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x  FOREWORD 

This  handbook  is  issued  under  the  authority  of  DoD  Instruction  3235.1,  "Test 
and  Evaluation  of  System  Reliability,  Availability,  and  Maintainability," 
February  1,  1982.  Its  purpose  is  to  provide  instruction  in  the  analytical 
assessment  of  System  Reliability,  Availability,  and  Maintainability  (RAM) 
performance. 

The  provisions  of  this  handbook  apply  to  the  Office  of  the  Secretary  of  Defense, 
the  Military  Departments,  the  Organization  of  the  Joint  Chiefs  of  Staff,  and  the 
Defense  Agencies  (hereafter  referred  to  as  "DoD  Components"). 

This  handbook  is  effective  immediately  and  may  be  used  on  an  optional  basis  by 
DoD  Components  engaged  in  system  RAM. 

Send  recommended  changes  to  the  handbook  to: 

Office  of  the  Director 

Defense  Test  and  Evaluation  OUSDRE/DDTE 

Washington,  D.C.  20301 

DoD  Components  may  obtain  copies  of  this  handbook  through  their  own  publication 
channels.  Other  federal  agencies  and  the  public  may  obtain  copies  from  the 

Naval  Publications  and  Forma  Canter,  5801  Tabor  Avenue,  Philadelphia, Fa.  19120. 


Director  Defense 
Test  and  Evaluation 
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MEMORANDUM  FOR  TEXT  USERS 

SUBJECT:  The  Application  of  Statistical  Concepts  to  Test  and  Evaluation 


Test  and  Evaluation  of  military  systems  and  equipment  is  conducted  by  DoD  to 
support  the  assessment  of  system  performance  characteristics.  These  assess¬ 
ments  are  an  integral  part  of  the  decision  process  inherent  in  the  acquisition 
cycle. 

In  many  hardware  and  software  development  programs,  testing  has  become  a 
controversial  issue.  Questions  which  often  arise  are:  How  much  testing  is 
enough?  Is  the  hardware/ software  ready  for  testing?  Are  hardware  require¬ 
ments,  assessment  parameters  and  critical  issues  adequately  defined?  Does  the 
test  effort  represent  the  minimum  time  and  resource  program  consistent  with 
meaningful  results?  Have  the  development  and  operational  testing  cycles  been 
integrated  so  as  to  form  an  efficient  evaluation  program?  And  so  on. 

This  text  presents  concepts  and  techniques  for  designing  test  plans  which  can 
verify  that  previously  established  system  suitability  requirements  have  been 
achieved.  We  realize,  of  course,  that  test  resource  availability  may  be 
adversely  affected  by  cost,  schedule  and  operational  urgency  constraints.  In 
such  cases,  alternate  test  plans  which  represent  the  most  meaningful,  timely 
and  cost  effective  approach,  consistent  with  these  constraints,  must  be  de¬ 
veloped.  In  any  event,  it  is  essential  that  ail  participants  understand  the 
critical  issues  being  addressed  and  the  acquisition  risks  inherent  in  con¬ 
ducting  a  limited  test  program. 

The  design  and  execution  of  sound  test  programs  is  NO  accident.  It  requires 
numerous  hours  of  research  and  planning  and  a  thorough  understanding  of  test¬ 
ing  techniques,  the  test  system  and  its  operating  scenario.  Further,  the  test 
results  must  support  the  development  of  realistic  performance  estimates  for 
the  entire  production  run,  after  having  tested  relatively  few  systems.  Herein 
lies  the  usefulness  of  the  statistical  concepts  contained  in  this  text. 

The  topics  addressed  in  this  text  *>ill  familiarize  the  reader  with  the 
statistical  concepts  relevant  to  test  design  and  performance  assessment.  In 
short,  these  topics,  when  combined  with  common  sense  and  technical  expertise 
formulate  the  basis  of  all  sound  test  programs. 


I&hasa  Linder 
Director  Defense 
Test  and  Evaluation 
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CHAPTER  1 


INTRODUCTION 


One  step  in  the  acquisition  of  ailitary  weapon  systems  and  equipment  ij  the 
verification  that  the  candidate  systems  do,  in  fact,  perform  in  accordance 
with  previously  specified  operational  requirements.  The  verification  process 
involves  the  design  of  test  programs  which  provide  an  adequate  data  base  to 
support  realistic  assessments  of  hardware  characteristics.  This  text  outlines 
the  concepts  and  techniques  to  be  used  in  structuring  such  test  programs  and 
analyzing  the  resulting  data . 

Since  verifying  the  performance  of  every  hardware  item  to  be  procured  is 
neither  practical  nor  possible,  we  base  our  projection  of  the  entire  popula¬ 
tion's  performance  on  the  assessment  of  an  available  sample.  This  sample  may 
consist  of  the  first  10  preproduction  aircraft  of  a  projected  buy  of  725  or  50 
percent  of  a  lot  of  high- reliability  pyrotechnic  devices.  In  either  case,  we 
are  utilizing  statistical  techniques  to  project  or  estimate  the  true  value  of 
some  population's  characteristic,  such  as  reliability,  maintainability, 
weight,  size,  etc. 

The  material  contained  in  the  following  chapters  is  designed  to  familiarize 
the  reader  with  the  various  statistical  concepts  and  techniques  required  to 
thoroughly  understand  the  relationships  among  test  design,  assessment  and 
projection  of  population  characteristics. 

The  beginning  chapters  present  background  material  on  the  three  basic  quanti¬ 
tative  system  parameters:  reliability,  availability  and  maintainability.  The 
various  definitions  of  these  basic  terms  are  discussed,  as  are  the  relation¬ 
ships  among  these  parameters.  The  statistical  concepts  of  confidence  and 
producer's  and  consumer's  risk  are  next  introduced,  with  the  majority  of  the 
text  devoted  to  the  practical  application  and  significance  of  these  concepts. 
The  chapters  utilize  a  combination  of  narrative  and  case  studies  to  introduce 
and  illustrate  the  usefulness  of  the  concepts.  It  will  prove  quite  useful  to 
refer  to  the  case  studies  while  reading  through  the  chapters.  This  study 
technique  will  prove  especially  helpful  in  Chapters  7  and  8,  which  present 
information  on  analyzing  test  data  and  structuring  statistically  adequate  test 
programs.  Chapter  9  contains  an  introductory  discussion  of  the  reliability 
growth  concept.  Chapter  10  presents  qualitative  aspects  of  test  planning 
aliug  with  a  description  of  data  collection  requirements. 

It  should  be  noted  that  there  is  no  intent  here  to  indicate  that  all  DoD  test 
programs  must  produce  statistically  significant  test  results.  Obviously  some 
will,  but  it  is  essential  to  understand  the  risk  associated  with  a  proposed 
test  program  and  the  confidence  associated  with  specific  results  before  recom¬ 
mending  a  course  of  action. 

A  first  glance  at  the  middle  of  one  of  the  more  "intense"  chapters  will 
quickly  bring  the  reader  to  the  conclusion  that  the  real  title  of  the  text 
should  have  been  "Everything  You  Never  Knew  about  Statistics  and  Never 
Will  "--ELAM! !  In  fact,  the  text  could  be  entitled  "What  To  Do  Until  the 
Statistician  Arrives."  Anyone  able  to  work  through  the  entire  text  without 


/ 


•ay  questions  needs  nc  static'iician — for  most  tasks.  The  text,  however,  makes 
no  attempt  to  eliminate  the  need  for  expert  advice  but  rather  attempts  to  aid 
the  reader  in  recognising  the  simplicity  of  some  concepts,  the  complexity  of 
others,  the  assumptions  and  limitations  associated  with  all  of  them,  and  the 
importance  of  the  topic  to  test  and  evaluation  in  general. 


CHAPTER  2 


RELIABILITY 


Reliability  is  a  tens  used  to  describe  quantitatively  how  failure-free  a 
system  is  likely  to  be  during  a  given  period  of  operation.  The  ability  to 
express  reliability  numerically  is  crucial,  because  it  enables  us  to  con¬ 
cretely  identify  the  user's  needs,  contractual  specifications,  test  guidelines 
and  performance  assessment. 

Reliability 

Reliability  is  defined  as  the  probability  that  an  item  will  perform  its  in¬ 
tended  function  for  a  specified  interval  under  stated  conditions.  This  def¬ 
inition  does  not  specifically  consider  the  effect  of  the  age  of  the  system. 

The  following  adaptation  is  useful  for  systems  that  are  repairable.  Reli¬ 
ability,  for  repairable  systems,  is  the  probability  that  an  item  will  perform 
its  intended  function  for  a  specified  interval,  under  stated  conditions,  at  a 
given  age,  if  both  corrective  and  preventive  maintenance  are  performed  in  a 
specified  manner. 

If  a  system  is  capable  of  performing  multiple  missions,  or  if  it  can  perform 
one  or  more  of  its  missions  while  operating  in  a  degraded  condition  or  if  the 
mission  test  profiles  represent  only  typical  usage,  then,  the  concept  of  a 
unique  mission  reliability  becomes  difficult  to  define.  In  such  cases,  it  is 
preferable  to  use  a  reliability  mcacure  that  is  not  based  solely  on  the  length 
of  a  specified  time  interval  but  rather  on  the  definition  of  a  specific  mis- 
cion  profile  or  set  of  profiles.  This  concept  is  illustrated  in  Case  Study 
2-7. 

The  mef  of  the  terms  "stated  conditions"  and  "specified  interval"  are 
important  to  the  understanding  of  reliability.  The  term  "stated  conditions" 
refers  to  the  complete  definition  of  the  scenario  in  which  the  system  will 
operate.  For  a  ground  combat  vehicle,  these  conditions  include  climatic 
conditions,  road  surface,  and  loads  that  would  be  experienced  during  a 
selected  mission  profile.  These  conditions  should  reflect  operational  usage. 
The  term  "specified  interval"  refers  to  the  length  of  the  mission  described  in 
a  mission  profile.  This  interval  may  include  multiple  factors.  For  example, 
an  air  defense  system  mission  profile  will  define  an  interval  containing  X 
rounds  fired,  Y  hours  of  electronics  on-time  and  Z  miles  of  travel.  For  a 
simpler  system,  say  an  air-burst  artillery  round,  the  interval  may  include  a 
single  event --round  detonation. 

Mean  Time  Between  Failures 

Mean  time  between  failures  (MTBF)  is  defined  as  the  total  functioning  life  of 
a  population  of  an  item  during  a  specific  measurement  interval,  divided  by  the 
total  number  of  failures  within  the  population  during  that  interval.  MTBF  can 
be  interpreted  as  the  expected  length  of  time  a  system  will  be  operational 
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between  failures.  The  definition  is  true  for  time,  cycles,  miles,  events,  or 
other  measure-of-life  units.  These  various  measure-of-life  units  permit  the 
MTBF  term  to  be  tailored  to  the  reliability  requirements  of  a  specific  system. 
Some  examples  of  this  tailoring  are: 

-  Mean  rounds  between  failure  (MRBF) 

-  Mean  miles  between  operational  mission  failure  (MMBOMF) 

-  Mean  time  between  unscheduled  maintenance  actions  (MTBUMA) 

-  Mean  rounds  between  any  maintenance  actions  (MRBAMA) 

Failure  Rate 


Failure  rate  is  defined  as  the  number  of  failures  of  an  item  per  measure-of- 
life  unit  (e.g.,  cycles,  time,  miles  or  events  as  applicable).  This  measure 
is  more  difficult  to  visualize  from  an  operational  standpoint  than  the  MTBF 
measure,  but  is  a  useful  mathematical  term  which  frequently  appears  in  many 
engineering  and  statistical  calculations.  As  we  will  see  in  later  chapters 
the  failure  rate  is  the  reciprocal  of  the  MTBF  measure,  or 

Failure  Rate  - 


There  are  two  very  different  system  reliability  design  objectives.  One  is  to 
enhance  system  effectiveness;  the  other  is  to  minimize  the  burden  of  owning 
and  operating  the  system.  The  first  objective  is  addressed  by  means  of  mis¬ 
sion  reliability,  the  second  by  means  of  logistics-related  reliability. 
Measures  of  mission  reliability  address  only  those  incidents  that  affect 
mission  accomplishment.  Measures  of  logistics-related  reliability  address  all 
incidents  that  require  a  response  from  the  logistics  system. 

Mission  Reliability 

Mission  reliability  is  the  probability  that  a  system  will  perform  mis«<i  >a 
essential  functions  for  a  period  of  time  under  the  conditions  stated  in  tte 
mission  profile.  Mission  reliability  for  a  single  shot  type  of  system,  i.e., 
a  pyrotechnic  device,  would  not  include  a  time  period  constraint.  A  system 
with  a  high  mission  reliability  has  a  high  probability  of  successfully  com¬ 
pleting  the  defined  mission. 

Measures  of  mission  reliability  address  only  those  incidents  that  affect 
mission  accomplishment.  A  mission  reliability  analysis  must,  therefore, 
include  the  definition  of  mission  essential  functions.  For  example,  the 
mission  essential  functions  for  a  tank  might  be  to  move,  shoot  and  communi¬ 
cate.  More  specific  requirements  could  specify  minimum  speed,  shooting  ac¬ 
curacy  and  communi cation  range. 

See  Case  Study  2-7. 
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Logistics  (Maintenancc/Supply)  Related  Reliability 

Logistics  related  reliability  measures,  as  indicated  above,  jaust  be  selected 
so  that  they  account  for  or  address  all  incidents  that  require  a  response  frost 
the  logistics  system. 

Logistics  related  reliability  may  be  further  subdivided  into  maintenance 
related  reliability  and  supply  related  reliability.  These  parameters  re¬ 
spectively  represent  the  probability  that  no  corrective  maintenance  or  the 
probability  that  no  unscheduled  supply  demand  will  occur  following  the  comple¬ 
tion  of  a  specific  mission  profile. 

The  mathematical  models  used  to  evaluate  mission  and  logistics  reliability  for 
the  same  system  may  be  entirely  different.  This  is  illustrated  in  Case  Study 
2-3. 


CLASSIFICATION 


An  understanding  of  the  relationships  existing  between  the  above  reliability 
measures  and  other  terms  is  essential  to  the  knowledgeable  application  of 
these  parameters.  Figure  2-1  illustrates  the  effects  of  these  relationships 
not  their  causes.  For  example,  system  failures  may  be  caused  by  the  hardware 
itself,  by  the  operator,  or  by  inadequate/ faulty  maintenance. 


FltNNS  t-i  RIUAIIUTY  INCIDENT  CLASSIFICATION 


Mission  Failures 


Mission  failures  are  the  loss  of  any  of  the  mission's  essential  functions. 
Along  with  system  hardware  failures,  operator  errors  and  errors  in  publica¬ 
tions  that  cause  such  a  loss  arc  included  in  this  region.  Mission  failures 
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are  related  to  mission  reliability  measures  because  they  prevent  complete 
mission  accomplishment. 


System  Failures 

System  failures  are  hardware  malfunctions:  they  may  or  may  not  affect  the 
mission's  essential  functions,  and  they  may  or  may  not  require  spares  for 
correction.  A  system  failure  generally  requires  unscheduled  maintenance  so 
system  failures  heavily  influence  maintenance-related  reliability. 


Unscheduled  Spares  Demands 


Unscheduled  spares  demands  are  used  to  evaluate  supply- related  reliability. 
All  unscheduled  spares  demands  require  a  response  from  the  supply  system,  so 
they  form  the  basis  for  evaluating  supply-related  reliability. 


System/Mission  Failures  Requiring  Spares 


System/mission  failures  that  require  spares  for  correction  are  the  most 
critical.  Mission,  maintenance  and  supply  reliabilities  are  affected,  and  the 
system  runs  the  risk  of  being  held  in  a  non-mission-ready  status  for  an  ex¬ 
tended  period  of  time  by  logistics  delays. 


Contractually  Chargeable  Failures 


Contract  requirements  are  often  established  for  the  subset  of  mission  failures 
and/or  system  failures  for  which  the  contractor  can  be  held  accountable. 
Normally  excluded  from  contractual  chargeability  are  such  failure  categories 
as:  operator  or  maintenance  errors;  item  abuse;  secondary  failures  caused  by 
another  (primary)  failure;  and  failures  for  which  a  "fix"  has  been  identified 
(but  not  incorporated  in  the  test  article  that  failed).  It  should  be  noted 
that,  in  operation,  all  failures  (in  fact,  all  unscheduled  maintenance  ac¬ 
tions)  are  relevant  regardless  of  contractual  chargeability,  and  should  there¬ 
fore  be  included  in  operational  evaluations. 


System  reliability  models  are  utilized  to  describe  visually  and  mathematically 
the  relationship  between  system  components  and  their  effect  on  the  resulting 
system  reliability.  A  reliability  block  diagram  or  structural  model  provides 
the  visual  representation  while  the  mathematical  or  "math"  model  provides  the 
analytical  tool  to  calculate  quantitative  reliability  values. 


The  following  notation  is  used  in  the  discussion  of  reliability  models: 


Rg  =  reliability  of  the  system 

R^  =  reliability  of  the  i^1  subsystem 

Q  =  1  -  R  =  unreliability  of  the  system 

8  S 

=  1  -  R^  =  unreliability  of  the  ifc^  subsystem 
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n  =  product  of  (Note:  This  operator  is  used  in  the  same  fashion  as 
£  for  summation,  but  it  indicates  multiplication  rather  than 
addition.) 

Note:  In  the  following  discussion  it  xs  assumed  that  all  subsystems  function 
independently  of  one  another,  that  is,  failures  of  different  subsystems  are 
statistically  independent  of  each  other.  For  many  systems  this  represents  a 
realistic  assumption.  The  reliability  analysis  for  dependent  subsystems  is 
significantly  more  complex.  Independent  operation,  practically  speaking, 
means  that  a  failure  of  one  system  will  not  cause  a  change  in  the  failure 
characteristics  of  one  or  more  other  subsystems.  Therefore,  replacement  of 
the  single  failed  subsystem  should  permit  continued  operation  of  the  entire 
system ,  because  other  subsystems  were  not  affected. 

Series  Model 


When  a  group '  of  components  or  subsystems  is  such  that  all  must  function  prop¬ 
erly  for  the  system  to  succeed,  they  are  said  to  be  in  series.  A  system 
consisting  of  a  series  arrangement  of  n  subsystems  is  illustrated  in  the 
following  block  diagram: 


The  mathematical  model  is 

n 

R  JSR.IL...H  =  II  R, .  (2.1) 

s  l  i  n  .  i 

i=l 

See  Case  Studies  2-1,  2-2,  2-3,  2-5,  and  2-6  for  examples  of  reliability 
series  models. 

Redundant  Models 

The  mission  reliability  of  a  system  containiug  independent  subsystems  can 
usually  be  increased  by  using  subsystems  in  a  redundant  fashion,  that  is, 
providing  more  subsystems  than  are  absolutely  necessary  for  satisfactory 
performance.  The  incorporation  of  redundancy  info  a  system  design  and  the 
subsequent  analysis  and  assessment  of  that  design  is  a  complex  task  and  will 
not  be  addressed  here  in  detail.  We  will  define  the  elements  of  redundancy 
and  present  several  simplified  examples. 

Redundancy  Characteristics .  Redundancy  can  be  defined  by  three  basic  char¬ 
acteristics. 

-  First,  the  level  at  which  redundancy  is  applied.  For  example,  we 
could  have  redundant  pieceparts,  redundant  black  boxes,  or  complete 
redundant  systems. 


-  Second,  the  operating  state  of  the  redundant  element.  The  redundant 
part,  subsystem,  etc.,  nay  exist  in  the  circuit  as  an  active  function¬ 
ing  element  or  as  a  passive,  power  off,  element.  For  example,  an 
airport  that  maintains  two  separate  operating  ground  control  approach 
radars  at  all  times  has  active  redundancy  for  that  capability.  Carry¬ 
ing  a  spare  tire  in  your  trunk  is  an  example  of  passive  redundancy. 

-  Third,  the  method  used  to  activate  the  redundant  element.  Consider 
the  passive  redundancy  case  of  the  spare  tire.  The  vehicle  driver 
represents  the  switching  device  that  decides  to  activate  the  spare. 
Obviously  mission  time  is  lost  in  installing  the  spare.  The  opposite 
case  is  represented  by  the  use  of  an  electronic  switching  network  that 
senses  the  failure  of  Box  A  and  automatically  switches  to  Box  B  with¬ 
out  lost  time  or  mission  interruption. 


FIGURE  2-2  PASSIVE  REDUNDANCY  WITH  AUTOMATIC  SWITCHING 


Our  examples  will  consider  only  simple  active  redundancy.  In  this  type  of  re¬ 
dundancy,  all  the  operable  subsystems  are  operating,  but  only  one  is  needed 
for  satisfactory  performance.  There  are  no  standby  subsystems,  and  no  repair 
is  permitted  during  the  mission.  Such  a  system  of  n  subsystems  is  illustrated 
in  block  diagram  fora  as: 


Note:  Simple  active  redundancy 
requires  that  only  one  of  the  n 
subsystems  be  operating  for 
mission  success. 
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The  mathematical  model  is 


Qs  =  «A- 


.q  =  n  q. 

u  i=i  1 


a 


n  (1-R.). 
i=l  1 


R  =  1-Q  =  1-  n  (1-R.).  (2.2) 
8  6  i=l  1 

This  mi  del  again  assumes  that  there  is  statistical  independence  among  failures 
of  the  subsystems.  This  assumption  is  important  because  dependence  between 
subsystems  can  have  a  significant  effect  on  system  reliability.  Calculations 
based  on  an  assumption  of  independence  can  be  erroneous  and  misleading.  In 
fact,  erroneously  assuming  failure  independence  will  often  result  in  overesti¬ 
mating  system  reliability  for  an  active  redundant  system  and  underestimating 
reliability  for  a  series  system. 

Implications  of  Redundant  Design.  While  redundant  design  does  improve  mission 
reliability,  its  use  must  be  weighed  against  the  inherent  disadvantages. 
These  disadvantages  include  greater  initial  cost,  increased  system  size  and 
weight,  increased  maintenance  burden  and  higher  spares  demand  rates.  These 
factors  must  be  considered  by  using  and  procuring  agencies  and  by  testing 
organizations  when  assessing  the  true  mission  capability  and  support 
requirements . 

Although  there  are  some  possible  exceptions,  redundancy  generally  improves 
mission  reliability  and  degrades  logistics  reliability.  Case  Study  2-3  gives 
a  numerical  comparison  between  mission-  and  maintenance- related  reliability. 

Mixed  Models 


One  system  configuration  that  is  often  encountered  is  one  in  which  subsystems 
are  in  series,  but  redundancy  (active)  is  applied  to  a  certain  critical  sub¬ 
system^).  A  typical  block  diagram  follows: 


This  type  of  model  (or  any  mixed  model,  for  that  matter)  is  characterized  by 
working  from  low  to  high  levels  of  assembly.  In  this  case,  assuming  inde¬ 
pendence  and  active  redundancy,  we  can  apply  equation  2.2. 

r4,5,6  =  i-a-v(1"R5)Cl"V  (2*3) 

We  can  now  represent  the  redundant  configuration  of  4,  5,  and  6  by  a  single 
block  on  the  diagram. 


We  can  now  apply  equation  2.1: 

Rs  ~  RiR2R3R4,5 ,6 

Rs  =  T'^2R3ll“(1'V(1“R5)(1"R6)1  (2.4) 

See  Case  Study  2*4  for  the  numerical  analysis  of  a  mixed  model. 

Functional  Models 


The  series,  redundant  and  mixed  models  mentioned  above,  are  hardware-oriented 
in  that  they  display  hardware  capabilities.  In  some  cases,  it  is  desirable  to 
model  a  system  from  a  functional  standpoint.  As  an  example,  the  functional 
reliability  block  diagram  for  a  tank  is  shown  below: 


The  functions  may  be  defined  as: 

-  MOVE.  The  mobility  system  must  be  capable  of  effectively  maneuvering 
such  that  the  system  can  maintain  its  assigned  position  within  a 
tactical  scenario.  Specific  requirements  are  determined  for  speed, 
turning,  terrain,  etc. 

-  SHOOT.  The  main  gun  must  be  capable  of  delivering  effective  fire  at 
the  rate  of  X  rounds  per  minute. 

-  COMMUNICATE.  The  communication  system  must  be  capable  of  providing 
two-way  communication  with  other  vehicles  and  with  fixed  stations 
within  specific  ranges  and  terrain  confines. 

Note  that  this  concept  addresses  mission-essential  functions,  but  in  no  way 
implies  how  these  functions  will  be  accomplished.  Generally  the  functional 
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Model  approach  is  helpful  in  the  program  formulation  stages  of  a  program  when 
specific:  hardware  information  is  not  necessary  and  frequently  not  desired. 
This  type  of  model  can  provide  a  useful  transition  from  operational  require¬ 
ment  to  engineering  specifications.  * 

Case  Study  2-7  utilizes  this  concept  to  evaluate  the  multi-mission,  multi-mode 
capabilities  of  a  system. 


The  previous  section  presented  the  topic  of  functional  reliability  models  and 
indicated  that  these  models  provided  a  useful  means  of  transitioning  from 
operational  requirement?  to  engineering  specifications.  The  process  of  trans¬ 
itioning  from  operational  requirements  to  engineering  specifications  is  known 
»»  r£li*.bility  allocation.  The  reliability  allocation  process  "allocates"  the 
reliability  budget  for  a  given  system  or  subsystem  to  the  individual  com¬ 
ponents  of  that  system  or  subsystem.  An  example  will  prove  helpful. 

Suppose  w  have  previously  determined  that  the  reliability  of  an  electronic 
subsystem  A,  must  equal  or  exceed  0.90,  and  that  this  subsystem  has  been 
designed  with  5  parts  all  functioning  in  series.  For  this  example,  we  will 
assume  Parts  1,2  and  3  are  the  same  and  the  best  available  piece  part  for 
Part  u  has  a  reliability  of  0.990.  How  can  we  allocate  the  reliability  budget 
for  tills  subsystem  to  its  individual  parts?  y  8 


SUBSYSTEM  A 


Using  equation  2.1  we  have 

^Total  =  R1R2R3R4R5 

=  RiR2R3(0*990>R5 
Solving  for  RjR^Rg  we  have 

R1R2R3R5  ~  6.990  =  °'909 
If  we  assume  Rj  -  R2  =  R3  =  R5  theQ» 


R.  =  R0  =  R  =  R  =  «/0. 909  s  0.976 


If  we  can  locate  piece  parts  for  Part  5  with  R,.  =  0.985,  then 


o  t)  -  2  •  909  0 . 909  q  g23  go 
*1T3  R5  0.985  b0» 


Rj  =  R2  =  R3  =  v0*923  =  0.973. 


Summary  of  Allocation 

Case  I  Case  II 

R1  =  R2  -  R3  =  r5  =  0.976  Rj  =  R2  =  R3  =  0.973 

R4  =  0.990  R4  =  0.990 

R5  =  0.985 

Another,  and  somewhat  more  frequently  used  approach  to  reliability  allocation 
is  one  in  which  reliability  is  allocated  on  the  basis  of  allowable  failures  or 
failure  rates. 


The  understanding  of  reliability  allocation  is  important  to  those  individuals 
who  must  be  concerned  with  hardware  operating  characteristics  below  the  system 
level.  This  is  especially  true  to  development  and  testing  organizations  who 
are  frequently  faced  with  predicting  system  performance  early  in  development, 
when  no  full-up  system  exists  but  when  subsystem  or  component  test  data  may  be 
available. 


See  Case  Study  2-5  for  another  example  of  reliability  allocation. 


CASE  STUDY  NO.  2-1 


Background 

A  system  is  composed  of  5  subsystems ,  each  of  which  must  succeed  for  system 
success.  Past  records  indicate  the  subsystem  reliabilities  to  be  as  shown  on 
the  diagram. 


Rx  =  0.9  R2  =  0.95  R3  =  0.99  R^  =  0.99  R5  =  0.9 


Determine 


System  reliability. 

Solution 

Applying  equation  2.1: 

Rs  =  RjR2R3R4R5  =  (0.9) (0.95) (0.99) (0.99) (0.9)  =  0.75 
Commentary 

Note  that  the  system  reliability  is  lower  than  that  of  the  worst  subsystem. 
This  is  generally  the  case  for  a  series  structured  system. 


CASE  STUDY  NO.  2-2 


Background 

An  electronic  ay Mints  has  1000  components  in  reliability  series.  The  reli¬ 
ability  of  each  component  is  0.999. 

Determine 

System  reliability. 

Solution 

Applying  equation  2.1: 

1000  inno 

R  =  *n  0.999  a  (i, .999)  S  0.368 
8  i=i 


Commentary 

1.  Even  though  a  component  reliability  of  0.999  sounds  good,  the  sheer  number 
of  these  components  causes  a  low  system  reliability. 

2.  Even  though  0.999  £  1.0,  the  difference  is  crucial.  For  high  reliability 
values,  the  probability  of  failure  often  gives  a  clearer  picture.  For 
example,  increasing  the  component  reliability  fro*  0.999  to  0.9999  re¬ 
quires  a  ten-fold  reduction  of  the  failure  probability. 

2.  Problems  such  as  this,  involving  large  powers,  are  solved  effcrMessly 
with  an  electronic  calculator  with  a  power  capability.  The  more  tradi¬ 
tional  approach  is,  of  course,  the  use  of  logarithms  and  anti-] ogaxithm 
tables . 


CASE  STUDY  SO.  2-3 


The  missiou  reliability  of  the  system  is  described  by  the  following  block 
diagram.  All  subsystems  are  identical  and  each  has  r  reliability  of  R  =  0.90. 
No  repairs  are  possible  during  a  aission,  but  will  be  Blade  following  missions 
in  which  failures  occur.  Failures  occur  independently.  For  this  case,  we 
assume  that  any  mission  failure  will  require  an  unscheduled  maintenance 
action. 


Determine 

System  mission  reliability  and  maintenance  reliability  (probability  of  no 
corrective  maintenance  following  a  mission). 


Solution 

System  mission  reliability:  Applying  equation  2.2: 


R  =  1-(1-R)n  =  l-(l-0.9)3  =  l-(O.l)3  =  1-0.001  =  0.999 

s 

Maintenance  reliability:  An  unscheduled  maintenance  action  required  by  any 
subsystem  is  chargeable  to  the  maintenance  burden  of  the  entire  system,  i.e., 
a  failure,  defined  in  this  case  to  be  a  requirement  for  corrective  maintenance 
of  one  subsystem,  is  charged  as  a  system  failure.  As  a  consequence,  we  model 
maintenance  reliability  for  this  system  using  a  series  structure. 


Commentary 


tf. 

I 

t 


1.  Note  that  we  evaluated  system  mission  reliability;  that  is,  the  reli¬ 
ability  of  the  hardware  alone. 

2.  Based  on  the  given  information,  it  is  apparent  that  a  system  consisting  of 
only  one  of  the  above  subsystems  will  have  a  probability  of  mission  fail¬ 
ure  equal  to  0.1  and  a  probability  of  corrective  maintenance  action  also 
equal  to  0.1.  The  system  with  triple  active  redundancy  has  a  mission 
reliability  of  0.999,  which  corresponds  to  a  probability  of  mission  fail¬ 
ure  equal  to  0.001  (a  100-fold  reduction).  The  same  system  has  a  mainte¬ 
nance  reliability  of  0.729  which  corresponds  to  a  probability  of  cor¬ 
rective  maintenance  action  equal  to  0.271  (approximately  a  3-fold  in¬ 
crease).  The  procuring  and  using  agencies  must  decide  whether  to  contract 
for  redundancy  and  how  much  to  require  based  on  consideration  of  these 
parameters . 

3.  Note  that  with  active  redundancy  the  system  reliability  is  generally 
greater  than  the  reliability  of  the  best  subsystem. 

4.  For  this  example,  we  stipulated  that  any  mission  failure  would  require  an 
unscheduled  maintenance  action.  The  reader  should  note  that  this  is  not 
always  the  case. 

5.  It  is  possible,  for  example,  to  experience  the  failure  of  one  of  two 
redundant  mission  critical  subsystems  and  still  successfully  complete  the 
mission.  After  successful  mission  completion,  an  unscheduled  maintenance 
action  would  be  necessary  to  repair/replace  the  defective  critical  re¬ 
dundant  subsystem. 


CASE  STUDY  NO.  2-4 


Subsystem  II  as 


Kjl  =  1-(1-R3)3 


Subsystem  III  as 


RIII  =  i-O-Vd-Rjj)  s  l-d-Il-d-R1)2]R2}(i-R3)3 

Then  the  system  becomes 


Rs  =  R4RIII  =  R4{i“U-U'(i-R1)2]R2}(i-R3)3} 

=  0.9Q{l-{l-[l-(l-0.80)2](0.95)}(l-0.70)3}  =  0.87S 


The  general  mathematical  model  is  often  of  more  use  than  the  numerical 
solution,  since  it  permits  evaluating  a  variety  of  alternatives. 


CASE  STUDY  NO.  2-5 


Background 

An  air  defense  system  comprises  a  weapon  subsystem,  a  vehicle  subsystem  and  a 
fire  control  subsystem.  One*  of  the  mission  profiles  for  the  total  system 
requires  firing  250  rounds,  traveling  10  miles,  and  operating  the  fire  control 
system  for  15  hours.  The  respective  subsystem  reliabilities  for  this  mission 
profile  are: 


=  0.95 


®VE  =  0-99 


he  =  0,90 

Determine 


The  system  is  to  have  its  reliability  improved  by  redesign  of  the  fire  control 
subsystem.  What  reliability  is  required  of  this  subsystem  to  attain  a  system 
reliability  of  0.90? 

Solution 


This  is  a  series  system,  so  equation  2.1  is  applied: 


Ro  = 


1RWERVrERfC 


Using  stars  to  represent  requirements: 

* 

*  Rg  0  QQ 

=  R^R^  =  (0.95)  (0.99J  =  0,957 

Commentary 

This  represents  a  very  simple  form  of  reliability  apportionment;  that  is, 
allocating  a  system- level  requirement  among  lower  levels  of  assembly. 


Determine 


What  reliability  is  required  of  the  fire  control  system  to  attain  a  system 
reliability  of  0.95? 


xl  -  RS  -  0.95 

*FC  ~  VvE  ~  (0-95)(0.99) 


1.01 


Since  reliability  cannot  exceed  one,  the  requirement  cannot  be  wet.  At  the 
very  best,  Rj.c  =  1.0,  which  would  yield: 

Rg  *  C  “  (0.95)(0.99)(1.0)  =  0.9405 

Commentary 

The  "you  can't  get  there  from  here"  solution  points  wt  that  more  than  just 
the  fire  control  system  must  be  improved  if  a  rellablity  of  0.95  is  to  be 
attained. 


CASE  STUDY  NO.  2-6 


Background 

An  electronic  system  currently  in  use  has  approximately  1000  series  com¬ 
ponents.  The  reliability  for  a  24-hour  mission  is  0.95.  A  proposed  system 
will  utilize  approximately  2000  similar  components  in  series.  Assume  a 
similarity  in  design  practices,  quality  of  manufacture,  and  use  conditions. 

Determine 

1.  What  is  the  average  part  reliability  for  the  current  system? 

2.  What  reliability  should  be  expected  of  the  proposed  system  for  a  24-hour 
mission? 

Solution  1 


The  "average"  component  reliability  can  be  found  by  applying  equation  2.1  to 
the  old  system: 

R„  =  (R.)1000  =  0.95 

O  1 

Ri  a  (0.95)0,001  =  0.9999487 
For  the  new  system: 

v  (0 .9999487) 2000  =  0.9025 


Solution  ') 


The  new  system  is  the  reliability  equivalent  of  two  of  the  old  systems  used  in 
reliability  series .  Applying  equation  2.1: 

Rg  =  (0.95) (0.95)  =  (0.95)2  -  0.9025 
Commentary 

This  type  of  prediction  based  on  parts  count  is  particularly  useful  for  evalu¬ 
ating  feasibility. 


CASE  STUDY  NO.  2-7 


Background 

A  design  for  a  new  long-range  ocean  reconnaissance/weapons  control  system  has 
been  proposed.  The  system  will  be  used  aboard  an  aircraft  whose  reliability 
is  not  considered  in  this  evaluation.  The  system  has  been  designed  to  ac¬ 
complish  six  specific  missions. 

These  missions  are  defined  in  the  table  below.  Due  to  size,  weight  and  power 
limitations,  the  hardware  elements  peculiar  to  each  mission  must  be  combined 
with  hardware  elements  peculiar  to  other  missions  in  order  to  form  a  complete 
mission  hardware  set. 

For  example,  as  depicted  in  the  table  below,  the  mission  hardware  set  to 
accomplish  Mission  E  is  a  combination  of  hardware  elements  3,  4,  and  5. 


Mission 


Mission 

Mission  Description 

Hardware  ! 

A 

Long-Range  A/C  Surveillance 

1 

B 

Lcng/Short-Rauge  Surface  Ship  Detection 

1,2 

C 

Area  Sea  State  Information 

1,3 

D 

Subsurface  Surveillance 

1,3,4 

E 

Long-Range  Terminal  Guidance  of  Ship 
Launched  Missiles 

3,4,5 

F 

Wide  Area  Weather  Data 

1,2, 3, 6 

Mission-Peculiar  Subsystem  Reliabilities 

Hardware 

Element:  1234 

5  6 

Hardware  Element  Reliability:  95%  93%  99%  91%  90%  95% 


All  missions  are  three  hours  in  length  and  require  the  operation  of  all  ele¬ 
ments  in  the  hardware  set  for  the  full  three  hours. 

The  mission-peculiar  hardware  can  support  several  missions  simultaneously. 
Determine 


1.  What  is  the  probability  of  successfully  completing  each  of  the  six  mis¬ 
sions? 

2.  What  is  the  probability  of  successfully  completing  all  six  missions 
during  a  three-hour  period? 
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Solution 

1.  Since  the  elements  function  in  a  series  relationship,  the  individual 
reliabilities  are  multiplied.  Hence, 

RA  =  Ri  =  0.95 

Rg  =  R  x  R2  =  0.95  x  0.93  =  0.88 

Rc  =  Rj  x  R3  =  0.95  x  0.99  =  0.94 

Rjj  =  Rj  x  R3  x  R4  =  0.95  x  0.99  x  0.91  =  0.85 

Rg  =  R3  x  R^  x  R5  =  0.99  x  0.91  x  0.90  =  0.81 

B  R  x  R  I  R  x  a  0.95  x  0.93  x  0.99  x  0.95  =  0,83 

2.  The  probability  of  successfully  completing  all  six  missions  during  a 
single  three-hour  period  is  determined  by  multiplying  together  the  individual 
hardware  element  reliabilities.  This  is  done  because  all  individual  hardware 
elements  must  function  throughout  the  thrte-hour  period  to  enable  all  missions 
to  be  completed  successfully. 

Note  that  the  probability  of  completing  all  six  missions  successfully  is  not 
correctly  calculated  by  multiplying  together  the  individual  mission  reliabil¬ 
ities  R^  through  R^.  This  approach  would  erroneously  take  individual  hardware 

element  reliability  into  account  more  than  once. 

P  yotai  =  Probability  of  successfully  completing  six  missions  during 
Mission  *  tttee'hour  Period 

P  Total  *  K1  *  R2  “  R3  *  R«  x  *5  x  R6 


Mission 


=  0.95  x  0.93  x  0.99  x  0.91  x  0.90  x  0.95 

=  0.68 


Commentary 

The  significant  point  illustrated  by  this  problem  is  that  the  reliability  of 
multi-miasion/multi-mode  systems  should  be  presented  in  terms  of  their  indi¬ 
vidual  mission  reliabilities.  This  is  a  useful  technique  because  it  permits 
us  to  evaluate  a  system's  development  progress  relative  to  its  individual 
capabilities  rather  than  its  total  mission  reliability  which  may  prove  less 
meaningful.  For  example,  if  we  assume  that  Missions  A  and  B  are  the  primary 
missions,  we  see  that  the  system  has  an  88%  chance  of  successfully  completing 
both  functions  during  a  three-hour  period.  However,  if  we  evaluate  Missions  A 
and  B  along  with  the  remaining  four  lower-priority  missions,  we  fund  that  our 
analysis  of  the  total  system  capability  is  far  different,  i.e.,  68%  chance  of 


1 ; 


success.  Consequently,  for  this  esse,  approval  to  proceed  with  system  devel¬ 
opment  would  likely  be  given  based  on  the  criticality  of  Missions  A  and  B  and 
the  relatively  good  probability  of  successfully  completing  Missions  A  and  B. 

In  summary,  for  aulti-mission/multi-mode  systems,  the  presentation  of  indi¬ 
vidual  mission  liabilities  provides  a  more  meaningful  picture  of  a  system's 
development  status  and  its  current  and  projected  combat  capabilities  as  these 
relate  to  primary  mission  achievement. 

NOTE:  If  the  individual  mission  completion  times  had  not  been  equally  con¬ 
strained  to  the  three-hour  time  period,  we  would  have  been  required  to 
use  the  more  sophisticated  techniques  presented  in  Chapter  5. 


. . *  w  i1  *■  rTT"  ' 
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CHAPTER  3 


MAINTAINABILITY 


INTRODUCTION 

Maintainability  and  reliability  are  the  two  major  system  characteristics  that 
combine  to  form  the  commonly  used  effectiveness  index — availability.  While 
maintainability  is  important  as  a  factor  of  availability,  it  also  merits 
substantial  consideration  as  an  individual  system  characteristic.  The  im¬ 
portance  of  this  parameter  in  the  national  defense  posture  becomes  even  more 
obvious  when  we  consider  that  at  least  one  branch  of  the  armed  services  spends 
one-third  of  its  budget  on  system  maintenance  activities. 

Several  aspects  of  system  maintainability  must  be  addressed  before  an  accurate 
assessment  can  be  undertaken.  First,  the  difference  between  maintainability 
and  maintenance  must  be  understood.  Maintainability  is  a  design  considera¬ 
tion,  whereas  maintenance  is  the  consequence  of  design.  The  maintenance 
activity  must  live  with  whatever  maintainability  is  inherent  in  the  design, 
that  is,  it  must  preserve  the  existing  level  of  maintainability  and  can  do 
nothing  to  improve  that  level.  Maintenance  is  therefore  defined  as  "all 
actions  necessary  for  retaining  a  hardware  item  in  or  restoring  it  to  an 
optimal  design  condition."  The  second  consideration  is  that  maintainability 
requirements  can  be  specified,  measured  and  demonstrated.  Unlike  reliability, 
detailed  *nd  quantitative  study  of  maintainability  was  not  initiated  until  the 
early  1950s.  Until  recently,  maintainability  often  was  viewed  as  a  "common 
sense"  ingredient  of  design.  It  is  now  seen  as  a  factor  of  the  design  process 
and  an  inherent  design  characteristic  that  is  quantitative  in  nature  and 
therefore  lends  itself  to  specification,  demonstration,  and  trade-off  analysis 
with  such  characteristics  as  reliability  and  logistics  support. 

DEFINITION  OF  TERMS  AND  CONCEPTS 

Maintainability 

Maintainability  is  defined  as  a  characteristic  of  design  and  installation. 
This  characteristic  is  expressed  as  the  probability  that  an  item  will  be 
retained  in,  or  restored  to,  a  specified  condition  within  a  given  period  if 
prescribed  procedures  and  resources  are  used. 

A  commonly  used  working  definition  states  that  Maintainability  is  a  design 
consideration.  It  is  the  inherent  characteristic  of  a  finished  design  that 
determines  the  type  and  amount  of  maintenance  required  to  retain  that  design 
in,  or  restore  it  to,  a  specified  condition. 

Maintenance 


This  term  is  defined  as  all  actions  required  to  retain  an  item  in.  or  restore 
it  to,  a  specified  condition.  This  includes  diagnosis,  repair  and  inspection. 
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Preventive  Maintenance 


This  term  is  defined  as  systematic  inspection,  detection  and  correction  of 
incipient  failures  either  before  they  occur  or  before  they  develop  into  major 
defects.  Adjustment,  lubrication  and  scheduled  checks  are  included  in  the 
definition  of  preventive  maintenance. 

Corrective  Maintenance 

This  term  is  defined  as  that  maintenance  performed  on  a  non-scheduled  basis  to 
restore  equipment  to  satisfactory  condition  by  correcting  a  malfunction. 


An  understanding  of  the  principal  elements  of  maintainability  is  essential  to 
the  assessment  planning  process.  Certain  elements  of  a  design  basically 
define  a  system's  inherent  maintainability  and  thus  determine  the  related 
maintenance  burden  and  affect  system  availability. 

It  is  apparent,  from  the  definition  of  maintainability,  that  the  ability  and 
need  to  perform  maintenance  actions  is  the  underlying  consideration  when 
assessing  maintainability.  The  factors  which  affect  the  frequency  with  which 
maintenance  is  needed  are  reliability  and  the  preventive  maintenance  schedule. 
Those  which  affect  the  ability  to  perform  maintenance  on  a  given  weapon  system 
may  be  broken  down  into  three  categories:  the  physical  design  of  the  system, 
the  technical  personnel  performing  the  maintenance  and  the  support  facilities 
required. 

The  consideration  of  maintenance  when  designing  a  system  is  not  new.  There 
have  been  very  successful  efforts  in  the  development  of  automatic  check  out 
and  design  for  accessibility,  etc.  What  is  new  is  the  emphasis  on  quanti¬ 
tative  treatment  and  assessment  which  results  in  a  complete  change  in  design 
philosophy,  design  approach  and  design  management.  In  the  past,  design  for 
"maximum"  or  "optimum"  reliability  and  maintainability  was  emphasized.  This 
resulted  in  "unknown"  reliability  and  maintainability.  New  techniques  permit 
us  to  bring  qualitative  design  judgments  into  an  area  of  quantitative  measure¬ 
ment.  We  can  thus  establish  quantitative  design  goals  and  orient  the  design 
to  specific  mission  thresholds,  not  to  "optimum"  or  "maximum"  goals.  Main¬ 
tainability  design  considerations  and  testing  intended  to  assess  system 
maintainability  characteristics  must  be  based  on  established  quantitative  re¬ 
quirements  (thresholds  and  goals).  In  addition  to  verifying  these  values,  the 
maintainability  test  and  evaluation  program  also  should  address  the  impact  of 
physical  design  features  and  maintenance  action  frequency  o.n  system  main¬ 
tenance  . 

Some  physical  design  features  affect  the  speed  and  ease  with  which  maintenance 
can  be  performed.  These  features  and  pertinent  questions  are: 

-  Accessibility:  Can  the  item  to  be  repaired  or  adjusted  be  reached 
easily? 

-  Visibility:  Can  the  item  being  worked  on  be  seen? 
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-  Testability:  Can  system  faults  be  detected  and  isolated  to  the  faulty 
replaceable  assembly  level? 

•  Complexity:  How  many  subsystems  are  in  the  system?  How  many  parts 
are  used?  Are  the  parts  standard  or  special-purpose? 

-  Interchangeability:  Can  the  failed  or  malfunctioning  unit  be  "swapped 
around"  or  readily  replaced  with  an  identical  unit  with  no  need  for 
recalibration? 

In  addition  to  the  listed  physical  design  factors,  the  frequency  with  which 
each  maintenance  action  must  be  performed  is  a  major  factor  in  both  corrective 
and  scheduled  or  preventive  maintenance.  Thus,  reliability  could  have  a 
significant  impact  on  corrective  maintenance,  and  such  design  features  as 
self-check-out,  reduced  lubrication  requirements  and  self-adjustment  would 
affect  the  need  for  preventive  maintenance. 

Personnel  and  human  factor  considerations  are  of  prime  importance.  These 
considerations  include  the  experience  of  the  technician,  training,  skill 
level,  supervision  required,  supervision  available,  techniques  used,  physical 
coordination  and  strength  and  number  of  technicians  and  teamwork  requirements. 

Support  considerations  cover  the  logistics  system  and  maintenance  organization 
required  to  support  the  weapon  system.  They  include  availability  of  supplies, 
spare  parts,  technical  data  (TOs  and  manuals),  built-in  test  equipment,  ex¬ 
ternal  test  equipment  and  required  tools  (standard  or  special)  and  servicing 
equipment . 


While  some  elements  of  maintainability  can  be  assessed  individually,  it  should 
be  obvious  that  a  true  assessment  of  sys-tem  maintainability  generally  must  be 
developed  at  the  system  level  under  operating  conditions  and  using  production 
configuration  hardware. 
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The  following  paragraphs  describe  the  various  mathematical  indices  used  to 
quantify  maintainability.  It  is  important  to  remember,  however,  that  these 
relationships  merely  categorize  data  derived  from  planned  testing.  For  main¬ 
tainability,  the  test  planning  phase  is  equal  in  importance  to  the  assessment 
phase.  Testing  that  does  not  adequately  demonstrate  the  effect  of  the  above 
physical  design  features  and  personnel  anej.  support  aspects  provides  data  that 
effectively  conceal  the  impact  of  these  critical  elements. 

Indices  used  to  support  maintainability  analysis  must  be  composed  of  measur¬ 
able  quantities,  must  provide  effectiveness-oriented  data  and  must  be  readily 
obtainable  from  operational  and  applicable  development  testing.  If  they  are, 
system  designers,  users  and  testers  can  evaluate  candidate  system  character¬ 
istics  and  logistics  and  maintenance  practices  more  precisely. 


Mean-Tiae-to-Repair  (MTTR)  or  Met 


MTTR  is  the  total  corrective  maintenance  down  time  accumulated  during  a  spe¬ 
cific  period  divided  by  the  total  number  of  corrective  maintenance  actions 
completed  during  the  same  period.  MTTR  commonly  is  used  as  an  on-equipment 
measure  but  can  be  applied  to  each  maintenance  level  individually.  The  MTTR 
considers  active  corrective  maintenance  time  only.  Because  the  frequency  of 
corrective  maintenance  actions  and  the  number  of  man-hours  expended  are  not 
considered  (clock  hours  are  used) ,  this  index  does  not  provide  a  good  measure 
of  the  maintenance  burden. 

Maximum-Time -to-Repair  (MaxTTR)  or  MmaxC 

MmaxC  is  the  maximum  corrective  maintenance  down  time  within  which  either  90 
or  95  percent  (as  specified)  of  all  corrective  maintenance  actions  can  be 
accomplished.  An  MmaxC  requirement  is  useful  in  those  special  cases  in  which 
there  is  a  tolerable  down  time  for  the  system.  Ideally,  we  would  like  to  be 
able  to  state  an  absolute  maximum,  but  this  is  impractical  because  there  will 
inevitably  be  failures  that  require  exceptionally  long  repair  times.  A  95tb 
percentile  MmaxC  specification  requires  that  no  more  than  5  percent  of  all 
corrective  maintenance  actions  take  longer  than  MmaxC. 

Maintenance  Ratio  (MR) 

MR  is  the  cumulative  number  of  man-hours  of  maintenance  expended  in  direct 
labor  during  a  given  period  of  time,  divided  by  the  cumulative  number  of 
end-item  operating  hours  (or  rounds  or  miles)  during. the  same  time.  The  MR  is 
expressed  at  each  level  of  maintenance  and  summarized  for  all  levels  of  main¬ 
tenance  combined.  Both  corrective  and  preventive  maintenance  are  included. 
Man-hours  for  off-system  repair  of  replaced  components  and  man-hours  for  daily 
operational  checks  are  included  for  some  classes  of  systems. 

Particular  care  must  be  taken  that  the  operating  hour  base  be  clearly  defined. 
For  example,  in  the  case  of  combat  vehicles,  either  system  operating  hours  or 
engine  hours  could  be  used.  ' 

The  MR  is  a  useful  measure  of  the  relative  maintenance  burden  associated  with 
a  system.  It  provides  a  cleans  of  comparing  systems  and  is  useful  in  determin¬ 
ing  the  compatibility  of  a  system  with  the  size  of  the  maintenance  organiza¬ 
tion. 

For  fielded  systems,  the  MR  is  useful  in  maintenance  scheduling.  Some  care 
must  be  exercised  in  relating  the  MR  to  maintenance  costs,  because  au  in-house 
maintenance  organization  will  have  a  fixud  labor  cost,  independent  of  the 
nuount  of  actual  use  of  the  system,  but  principally  fixed  by  the  size  of  the 
maintenance  staff. 

Mean-Time-Between-Maintenance-Actions  (MTBMA) 

MTBMA  is  the  mean  of  the  distribution  of  the  time  intervals  between  either 
corrective  maintenance  actions,  preventive  maintenance  actions  or  all  mainte¬ 
nance  actions.  This  index  is  frequently  used  in  availability  calculations  and 
in  statistically-oriented  maintenance  analyses . 
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Average  Number  of  Technicians  Requ?  iced 


The  average  number  of  technicians  required  at  each  maintenance  vel  provides 
a  quantitative  means  of  expressing  the  personnel  aspects  of  t  overall  main¬ 
tenance  concept.  This  index  also  provides  t  conversion  factor  from  active 
down  time  to  labor  hours. 

Off-System  Maintainability  Indices 

The  indices  MTTR,  MoaxC  and  MR  all  specifically  exclude  off-system  maintenance 
actions.  Off-system  measures  are  particularly  important  if  a  system's  mainte¬ 
nance  concept  involves  extensive  use  of  modular  removal  and  replacement,  since 
this  type  of  concept  transfers  the  maintenance  burden  to  off-system  mainte¬ 
nance.  As  an  assessment  tool,  off-system  maintainability  measures  are  es¬ 
sential.  Without  them,  it  is  not  possible  to  assess  the  ability  of  combat 
environment  off-system  repair  and  logistics  capability  to  maintain  the  system. 
Because  of  the  system-peculiar  nature  of  these  parameters,  none  are  specified 
here.  Suffice  it  to  say  that  a  complete  set  of  on-  and  oif-system  indices  is 
required  to  adequately  assess  system  maintainability  and  total  maintenance 
burden. 

Annual  Support  Cost  (ASC) 

This  is  the  direct,  annual  cost  of  maintenance  personnel,  repair,  parts  and 
transportation  for  all  corrective  (either  on-system,  off-system  or  both)  and 
preventive  maintenance  actions  when  the  system  operates  X  hours  per  y^ar 
during  the  Nth  year  of  M  years  service  life,  where  the  system  is  defined  as  Y 
units  of  item  A,  Z  units  of  item  B,  etc. 

The  ASC  provides  another  means  of  quantifying  the  maintenance  burden  of  a 
system.  The  unique  feature  of  the  ASC  measure  is  the  recognition  that  mainte¬ 
nance  requirements  may  not  be  uniform  over  the  life  of  a  system.  For  example, 
a  combat  vehicle  will  experience  a  high-cost  year  when  its  engine  requires 
replacement  or  overhaul.  This  measure  provides  a  means  of  interrelating 
durability  requirements  and  policies  for  scheduled  maintenance. 

Case  Study  No.  3-1  illustrates  the  use  of  several  maintainability  indices. 


Introduction 

One  aspect  of  maintainability  that  has  received  significant  attention  in 
recent  system  designs  is  the  use  of  automatic  diagnostic  systems.  These 
systems  include  both  internal  or  integrated  diagnostic  systems,  referred  to  as 
built -ia-tBst  (BIT)  or  built-in-test-equipment  (BITE),  and  external  diagnostic 
systems,  referred  to  ;  automatic  test  equipment  (ATE),  test  sets  or  off-line 
test  equipment.  The  following  discussion  will  focuc  on  BIT  but  most  of  the 
key  points  apply  equally  to  other  diagnostic  systems. 
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Need  for  Automatic  Diagnostic  Systems  -  BIT 


As  technology  advances  continue  to  increase  the  capability  and  complexity  of 
modern  weapon  systems,  we  are  relying  more  on  the  use  of  automatic  diag¬ 
nostics,  i.e.,  BIT,  as  a  means  of  attaining  the  required  level  of  failure 
detection  capability.  Our  need  for  BIT  is  driven  by  operational  availability 
requirements  which  do  not  permit  the  lengthy  MTTRs  associated  with  detecting 
and  isolating  failure  modes  in  microcircuit  technology  equipment.  We  also 
find  that  because  BIT  operates  within  the  basic  system  and  at  the  same  func¬ 
tioning  speed,  it  therefore  affords  us  the  capability  to  detect  and  isolate 
failures  which  conventional  test  equipment  and  techniques  could  not  provide. 
Finally,  a  well  designed  BIT  system  can  substantially  reduce  the  need  for 
highly  trained  field  level  maintenance  personnel  by  permitting  less  skilled 
personnel  to  locate  failures  and  channel  suspect  hardware  to  centralized 
intermediate  repair  facilities  which  are  equipped  to  diagnose  and/or  repair 
defective  hardware. 

As  we  shall  discuss,  BIT  is  not  a  comprehensive  solution  to  all  system  main¬ 
tenance  problems  but  rather  a  necessary  tool  required  to  deal  with  the  com¬ 
plexity  of  modern  electronic  systems 

Specifying  BIT  Performance 

One  of  the  more  complex  tasks  inherent  in  the  acquisition  of  modern  systems  is 
the  development  of  realistic  and  meaningful  operational  requirements  and  their 
subsequent  conversion  into  understandable  and  achievable  contractual  speci¬ 
fications.  This  situation  is  equally  applicable  to  BIT.  Before  discussing 
this  topic  in  more  detail,  we  will  present  typical  performance  measures  or 
figures  of  merit  which  are  used  to  specify  BIT  performance. 

Percent  Detection.  The  percent  of  all  faults  or  failures  that  the  BIT  system 
detects . 

Percent  Isolation.  The  percent  of  detected  faults  or  failures  that  the  system 
will  isolate  to  a  specified  level  of  assembly.  For  example,  the  BIT  might 
isolate  to  one  box  or  to  three  or  less  printed  circuit  boards  in  a  box. 

Automatic  Fault  Isolation  Capability  (AFIC).  The  AFIC  is  the  product  of  per¬ 
cent  isolation  times  percent  detection. 

AFIC  =  %  detection  x  %  isolation 

Percent  of  False  Alarms.  The  percent  of  the  BIT  indicated  faults  where,  in 
fact,  no  failure  is  found  to  exist. 

Percent  of  False  Removals.  The  percentage  of  units  removed  because  of  BIT 
indications  which  are  subsequently  found  to  test  "good"  at  the  next  higher 
maintenance  station. 

For  each  of  the  above  parameters,  there  is  a  considerable  span  of  interpreta¬ 
tion.  For  '“xample,  does  the  percent  detection  refer  to  failure  modes  or  the 
percentage  of  all  failures  that  could  potentially  occur?  Does  the  detection 
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capability  apply  across  the  failure  spectrum,  i.e.,  mechanical  systems,  in¬ 
strumentation,  connections  and  softare,  or  is  its  diagnostic  capability  ap¬ 
plicable  only  to  electronic  hardware  systems? 


A  major  contractual  issue  relates  to  the  definition  of  failure.  Should  BIT 
performance  be  viewed  in  terms  of  "BIT  addressable"  failures ,  which  normally 
exclude  cable/connector,  etc.,  problems  as  not  contractually  chargeable,  or  in 
terms  of  all  operationally  relevant  maintenance  actions? 


An  important  consideration  relates  to  exactly  what  failures  BIT  can  detect. 
Our  BIT  system  will  operate  ineffectively  if  the  80%  of  detectable  failures 
occur  infrequently  while  the  remaining  20%  occur  with  predictable  regularity. 
It,  therefore,  becomes  important  to  specify  BIT  performance  measures  in  rela¬ 
tion  to  overall  mission  availability  requirements. 


Relative  to  isolation  characteristics,  will  the  BIT  isolate  failures  while  the 
basic  system  is  in  an  operational  mode,  or  must  the  basic  system  be  "shut 
down"  to  permit  the  isolation  software  package  to  be  run?  How  does  this 
characteristic  impact  mission  requirements?  Also,  to  what  "level"  will  the 
BIT  system  isolate  failures?  Early  BIT  systems  were  frequently  designed  to 
fault  isolate  to  the  module  level.  This  resulted  in  BIT  systems  as  complex 
as,  and  frequently  less  reliable  than,  the  basic  system.  The  current  trend  is 
to  isolate  to  the  subsystem  or  box  level  based  on  BIT's  ability  to  detect  ab¬ 
normal  output  signal  patterns.  Intermediate  and  depot  level  maintenance 
facilities  will  frequently  use  BIT  or  external  diagnosic  equipment  to  isolate 
to  the  board  or  piece-part  level. 


The  percent  of  false  alarms  is  a  difficult  parameter  to  measure  accurately 
because  an  initial  fault  detection  followed  by  an  analysis  indicating  that  no 
fault  exists  can  signify  several  different  occurrences,  such  as: 


The  BIT  system  erroneously  detected  a  fault. 


-  An  intermittent  out-of-tolerance  condition  exists — somewhere. 


-  A  failure  exists  but  cannot  be  readily  reproduced  in  a  maintenance 
environment. 


The  percent  of  false  removals  can  be  a  more  difficult  problem  to  address. 
False  removals  may  be  caused  by: 


-  Incorrect  BIT  logic. 


-  Wiring  or  connection  problems  which  manifest  themselves  as  faulty 
equipment . 


Improper  match  of  tolerances  between  the  BIT  and  test  equipment  at  the 
next  higher  maintenance  level. 


The  resolution  of  each  type  of  false  alarm  and  false  removal  requires  a  sub¬ 
stantially  different  response.  From  a  logistic  viewpoint,  false  alarms  often 
lead  to  false  removals  creating  unnecessary  demands  on  supply  and  maintenance 
systems.  Of  potentially  more  concern  is  the  fact  that  false  alarms  and 
removals  create  a  lack  of  confidence  in  the  BIT  system  to  the  point  where 
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maintenance  or  operations  personnel  may  ignore  certain  faelt  detection  indica¬ 
tions.  Under  these  conditions,  the  BIT  system  in  particular  and  the  mainte¬ 
nance  concept  in  general  canaot  mature  nor  provide  the  support  required  to 
Met  mission  requirements. 

The  specification  of  BIT  performance  must  be  tailored  to  the  specific  system 
under  consideration  as  veil  as  the  available  funds  and,  most  importantly,  the 
overall  mission  requirements.  This  tailoring  activity  must  include  a  compre- 
.hensive  definition  of  BIT  capability  based  upon  the  figures  of  merit  presented 
above . 

Characteristics  External  to  BIT 

There  are  two  important  considerations,  external  to  BIT,  which  must  be  ad¬ 
dressed  in  any  discussion  of  BIT  and  diagnostics.  First,  reliable  performance 
of  the  weapon  system  determines,  to  a  large  extent,  the  criticality  of  BIT 
performance.  If  the  b'-ic  system  is  very  reliable,  more  than  expected,  a 
shortfall  in  the  BIT  pi  .ormance  may  have  very  limited  impact  on  the  system's 
operational  utility.  Second,  it  must  be  remembered  that  generally  all  system 
faults  that  are  correctable  by  maintenance  action  must  eventually  be  detected 
and  isolated.  Therefore,  the  techniques,  tools,  manuals,  test  equipment  and 
personnel  required  to  detect  and  isolate  non-BIT  detectable  faults  can  be  a 
major  maintenance  consideration. 

The  following  example  illustrates  the  impact  of  BIT  on  the  overall  maintenance 
effort.  It  further  attempts  to  illustrate  the  effect  of  external  factors  on 
BIT  performance. 

Case  Description.  An  attack  aircraft  radar  is  composed  of  five  line  replace- 
able  traits (LRUs)  with  the  following  BIT  and  system  performance  character¬ 
istics. 

System;  Five  (5)  LRUs 

MTTR  (v/BIT) ;  2  hours  (includes  failures  which  have  been 

both  detected  and  isolated) 

MTTR  (no/BIT):  5  hours  (includes  failures  which  have  not 

been  isolated  but  may  have  been  detected) 

MFHBF:  50  flying  hours 

Time  Period  of  Interest:  2500  flying  hours 

BIT  Specified;  Percent  detection  =  90% 

Percent  isolation  =  90%  (to  the  LRU  level) 

False  alarm  rate  =  5%  (of  all  BIT  indications) 

NOTE:  Operating  time  is  assumed  to  be  flight  time  for  this  example. 

Before  beginning  the  analysis  of  th>  system,  note  that  we  have  specified  a 
relatively  high  capability  BIT  system  An  off-handed  examination  would  likely 
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conclude  that  with  this  extensive  BIT  coverage  there  is  minimal  additional 
isaintenance  action  required. 


-  How  many  total  failures  will  be  experienced  (on  the  average)  during 
the  2500  flying  hours? 

2500  total  hours  *  50  mean  hours  hetween  failures  -  50  failures 

-  How  many  of  these  failures  (on  the  average)  will  BIT  detect? 

50  failures  x  90%  =  45  BIT  detected  failures 

How  many  detected  failures  (on  the  average)  will  be  isolated  to  an 
LRU? 

45  detected  failures  x  90%  isolation  £  40  failures 

-  What  is  the  Automatic  Fault  Isolation  Capability  (AFIC)? 

AFIC  ~  %  detection  x  %  isolation  (LRU  level) 

=  0.9  x  0.9  »  0.81  =  81% 

-  How  many  false  alarm  indications  are  expected  to  occur  during  the  2500 
flight  hours? 

Total  BIT  indications  =  true  failure  detections  +  false  alarms 

=  x 

and, 

x  =  (BIT  detection  rate) (total  failures) 

+(false  alarm  rate) (total  BIT  indications) 

x  -  (0.90) (50)  +  (0.05)(x) 

(l-0.05)x  -  45 

x  -  47.36 

Therefore, 

False  Alarms  «  total  BIT  indications  -  true  indications 
=  47.36  -  45 
=  2.36 
£  2 
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What  is  the  total  corrective  maintenance  time  (on  the  average)  re¬ 
quired  to  repair  the  40  deteeted/isolated  failures? 


Time  =  40  failures  x  2  hours  (MTTR  w/BIT)  =  80  hours 

-  What  is  the  total  corrective  maintenance  time  (on  the  average)  re¬ 
quired  to  repair  the  remaining  10  no/BIT  detected/isolated  failures? 

Time  -  10  failures  x  5  hours  (MTTR  no/BIT)  =  50  hours 

-  If  we  assume  that  manual  or  i.'/BIT  maintenance  time  is  required  to 
resolve  the  false  alarm  indications,  what  total  no/BIT  corrective 
maintenance  time  is  required  fos  the  2500  flying  hour  period? 

Total  (no/BIT)  time  =  failure  repair  time 

+  false  alarm  maintenance  time 

*  (10) (5)  +  (2) (5)  =  60  hours 

-  What  is  the  total  corrective  maintenance  time  M.  anticipated  during 

the  2500  hours?  t 

-  BIT  maintenance  +  no/BIT  maintenance  =  80  +  60  =  140  hours 

-  Note  that  even  with  a  relatively  high  AFIC  of  81%  the  no/BIT  oriented 
corrective  maintenance  represents  43%  of  the  total  anticipated  cor¬ 
rective  maintenance  hours. 

-  Note  that  we  have  not  considered  the  impact  of  any  scheduled/ 
preventive  maintenance  for  our  system.  This  additional  maintenance  is 
generally  not  associated  with  BIT. 

The  information  presented  in  this  example  is  greatly  simplified  in  that  we 
have  ignored  many  of  the  pitfalls  and  controversial  areas  discussed  in  the 
previous  sections.  Also  1  e  that  we  are  basically  dealing  with  planning  type 
information  in  that  vt  are  assuming  that  the  BIT  AFIC  will  be  81%.  If,  in 
fact,  the  AFIC  is  81%  theu  57%  of  the  maintenance  effort  will  be  oriented 
toward  BIT  detected/isolated  failures.  If  tue  true  AFIC  is  found  to  be  lower, 
it  will  be  necessary  to  reevaluate  the  overall  effectiveness  of  the  entire 
maintenance  and  logistics  programs  as  well  as  total  mission  effectiveness. 
Our  next  section  discusses  some  of  the  difficulties  involved  in  the  design  and 
evaluation  of  a  BIT  system  which  must  perform  in  accordance  with  established 
specifications . 

Basic  System/BIT  Development  and  Evaluation  Considerations 

The  development  and  evaluation  of  BIT  and  diagnostics  has  traditionally  been 
an  activity  that  has  chronologically  followed  basic  system  efforts.  The  argu¬ 
ment  usually  presented  is  th«t  "the  basic  system  has  to  be  designed  and  evalu¬ 
ated  before  we  know  what  the  BIT  is  suppose  to  test."  This  argument  has  some 
basis  in  fact,  but  there  are  significant  drawbacks  associated  with  lengthy 
schedule  differentials  between  basic  system  and  BIT  design  and  testing.  For 
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example,  design  considerations  relating  to  the  basic  system  6uch  as  partition¬ 
ing  and  subsystem  layout  determine  to  a  large  extent  the  required  BIT  design. 
The  BIT  design  is  also  driven  by  the  prediction  and  occurrence  of  basic  system 
failures  modes  which  BIT  is  expected  to  address.  Consequently,  the  two  design 
efforts  cannot  be  conducted  in  isolation  from  one  another. 

From  an  evaluation  viewpoint,  conducting  the  BIT  evaluation  after  the  basic 
system  tests  are  completed  may  preclude  BIT  improvement  options  from  being  in¬ 
corporated  because  the  already  finalized  basic  system  desipn  would  be  sub¬ 
stantially  impacted.  Likewise,  an  inadequate  evaluation  of  BIT  which  leads  to 
an  erroneous  confirmation  of  its  capabilities  (AF1C)  will  result  in  a  sub¬ 
stantial  impact  to  system  operational  effectiveness. 

Determination  of  Basic  System  Failure  Modes  and  Frequency  of  Occurrence 

The  design  of  BIT  is  based  upon  two  assumptions  regarding  the  reliability  of 
the  basic  weapon  system:  accurate  identification  of  failure  modes  and  correct 
estimation  of  the  frequency  of  occurrence  of  the  failure  modes.  If  either  of 
these  assumptions  is  proven  incorrect  by  test  or  operational  experience,  the 
resultant  BIT  performance  is  likely  to  be  inadequate  or  at  least  less  effec¬ 
tive  than  anticipated.  The  following  two  situations,  based  on  the  previous 
example,  will  illustrate  the  impact  of  these  two  assumptions. 

% 

-  Situation  1.  An  unforeseen  failure  node  is  observed  iu  the  radar 
system  every  250  flying  hours.  What  impact  does  this  have  on  the 
uo/BIT  maintenance? 

New  failures  =  2500  flying  hours  x  1  failure  per  250  hours 
-  10  failures  (new). 

Maintenance  time  associated  with  uo/BIT  detected  failures 
=  10  x  5  hours/ failure 
=50  hours. 

Total  Maintenance  hours  =  80  +  60  +  50  =  190  hours. 

Total  no/BIT  maintenance  =  60  +  50  =  110  hours. 

This  represents  58%  of  total  maintenance. 

BIT  (detected/isolated)  maintenance  -  80  hour®  =  42%  of  total. 

This  represents  42%  of  total  maintenance. 

Note  that  the  discovery  of  one:  unforeseen,  no/ BIT  detectable  failure 
has  a  relatively  significant  impact  on  the  comparable*  magnitude  of  the 
two  maintenance  percentages. 


TABLE  3-1.  PERCENT  OF  TOTAL  MAINTENANCE  HOURS 


BIT 

No/BIT 

Previous  Estimate 

57% 

43% 

Current  Estimate 
(including  new  failure) 

42% 

58% 

-  Situation  2.  One  of  the  original  BIT  detectable  failures  was  pre¬ 
dicted  to  have  a  very  low  frequency  of  occurrence.  BIT  detection  for 
this  failure  was  considered  unnecessary  and  was,  therefore,  not  in¬ 
cluded  in  the  original  BIT  design  to  detect  90%  of  the  failures.  It 
is  now  found  that  the  failure  occurs  five  times  as  often  as  expected. 
This  is  a  realistic  situation  and  one  which  again  directly  impacts  the 
no/EIT  maintenance  man-hours. 

•* 

Test  and  Evaluation  of  BIT  Systems.  The  test  and  evaluation  of  BIT  systems 
and  the  reliable  prediction  of  BIT  performance  are  areas  of  controversy.  The 
following  paragraphs  present  some  of  the  factors  supporting  this  statement. 

BIT  systems  are  hardware/ software  logic  networks  designed  to  detect  the 
presence  of  an  unwanted  signal  or  the  absence  of  a  desired  signal ,  each  rep¬ 
resenting  a  failure  mode.  Each  failure  mode  is  detected  by  a  specific  logic 
network  tailored  to  detect  a  specific  failure.  While  the  same  network  may  be 
designed  to  detect  a  specific  failure  in  several  components,  there  is  no 
assurance  that  the  logic  is  correct  until  verified  by  test.  It  is  possible  to 
project,  using  statistical  techniques,  BIT  performance  assuming  we  have  a 
large  enough  representative  sample  of  failures. 

Unlike  reliability  testing  which  has  matured  over  the  past  40  years,  BIT 
testing  and  BIT  system  design  represent  less  mature  technologies  and  are  just 
now  beginning  to  receive  increased  management  emphasis.  This  lack  of  maturity 
and  attention  has  resulted  in  circumstances  which  are  not  conducive  to  gather¬ 
ing  an  adequate  siase,  representative  data  base  needed  to  support  accurate  and 
dependable  estimates  of  BIT  performance  at  decision  milestones.  A  lack  of 
confidence  in  BIT  performance  assessments  has  resulted  because  of  these  cir¬ 
cumstances  . 

Since  we  are  not  content  nor  have  the  time  to  just  wait  for  the  basic  system 
to  experience  random  failures,  we  decide  to  "cause"  failures  using  "synthetic 
fault  insertion."  These  faults  are  generally  selected  from  a  contractor- 
provided  list  cf  possible  faults--all  of  which  are  thought  to  be  detectable. 
Wo  insert  these  faults  and  BIT  detects  and  isolates  93%  of.  those  inserted. 
This  does  not  mean  that  we  have  a  93%  AFIC  BIT  system.  Why?  Because  the  data 
is  not  a  representative  random  sample  of  the  entire  failure  population  and, 
therefore,  cannot  be  used  to  make  statistically  valid  predictions  of  future 
performance. 

While  synthetic  fault  insertion  has  recognised  limitations  in  predicting 
future  operational.  BIT  performance,  it  is  a  valuable  and  necessary  tool  during 
the  engineering  development  phase.  Also,  fault  insertion  can  be  used  to 
simulate  random  failures  which  we  ■'nor  will  occur  but  as  yet  have  not  been 
detected  during  I)T  or  0T  testing.  These  include  problems  such  as  feulty 
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connector  and  circuit  board  plAg-in  points  as  well  as  the  effects  of  poor 
maintenance  or  rough  handling. 

Because  of  the  lack  of  system  mcturity  (especially  software)  and  the  required 
use  of  fault  insertion,  we  find  that  there  is  normally  insufficient  good  data 
available  to  support  early,  accurate  and  defendable  estimates  of  BIT  perform¬ 
ance.  It  has  generally  required  a  few  years  of  operational  exposure  to  de¬ 
velop  an  adequate  data  base  to  support  a  BIT  performance  analysis. 

Current  trends  support  early  reliability  testing  during  development  to  facil¬ 
itate  identification  of  failure  modes  and  timely  incorporation  of  design 
improvements.  These  tests  provide  a  data  base  to  support  preliminary  esti- 
mates  of  system  reliability.  What  is  most  frequently  overlooked  is  that  this 
data,  after  minimal  screening,  could  also  be  used  to  monitor,  verify,  and 
upgrade  BIT  performance,  assuming,  of  course,  that  the  BIT  system  is 
functional  at  this  stage  in  development.  This  action  requires  a  disciplined 
approach  toward  the  use  of  BIT  in  failure  detection  early  in  the  development 
cycle  which  has  not  been  prevalent  in  previous  programs. 

In  summary,  there  is,  and  will  remain,  a  requirement  to  assess  BIT  performance 
during  the  system  development  and  Initial  Operational  Test  and  Evaluation 
(IOT&E)  phase.  The  developing  and  testing  organizations  must  support  this 
assessment  using  all  available  data.  This  includes  combining  random  failure 
detection  data  with  data  from  contractor  demonstrations  and  fault  insertion 
trials.  Early  emphasis  on  BIT  design  will  generally  result  in  accelerated  BIT 
system  maturity  and  more  accurate  early  projections  of  BIT  performance.  BIT 
assessment  should  be  actively  pursued  throughout  the  deployment  phase  to 
assure  that  required  software  and  hardware  changes  are  incorporated. 
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CASE  STUDY  NO.  3-1 


Background 

A  system  has  a  mean  time  to  repair  (MTTR)  of  30  minutes,  and  a  mean  time 
between  unscheduled  maintenance  actions  (MTBUMA)  of  SO  hours.  The  intended 
utilization  (actual  hours  of  operation)  of  the  system  is  5,000  hours  per  year. 

Determine 


1.  How  many  unscheduled  maintenance  actions  are  to  be  expected  each  year? 

2.  How  many  clock-hours  of  unscheduled  maintenance  are  to  be  expected  each 
year? 


'3.  If  an  average  of  1.5  technicians  is  required  to  perform  unscheduled 
maintenance,  how  many  corrective  maintenance  man-hours  are  required  each  year? 

4.  Ten  hours  of  scheduled  maintenance  are  required  every  1,000  hours  of 
operation.  Scheduled  maintenance  requires  only  one  technician.  What  is  the 
maintenance  ratio  (MR)  for  this  system? 

Solution 


MTTR  =  30  minutes 
MTBUMA  =  50  hours 
5,000  hours/year 

1.  Unscheduled  maintenance  actious  =  -  100/year 


2.  30  minutes  x  100  =  3000  minutes  -  50  hours 
50  hours  mean  repair  time/year 

3.  1.5  x  50  =  75  man-hours/year 

4.  Scheduled  maintenance  =  10  x  5  =  50  man-hours/year 


Hg  -  maintenance  man-hours 
operating  hours 


MR  * 


50  +  75 
5000 


MR  =  0.025 


CHAPTER  4 
AVAILABILITY 


1 


mmmm 

Availability  is  the  parameter  that  translates  system  reliability  aud  maintain¬ 
ability  characteristics  into  an  index  of  effectiveness.  It  is  based  on  the 
question,  "Is  the  equipment  available  in  a  working  c  mdition  when  it  is 
needed?"  The  ability  to  answer  this  question  for  a  sptcific  system  represents 
a  powerful  concept  in  itself,  and  there  are  additional  side  benefits  that 
result.  An  important  benefit  is  the  ability  to  use  the  availability  analysis 
as  a  platform  to  support  both  the  establishment  of  reliability  and  maintain¬ 
ability  parameters  and  trade-offs  between  these  parameters.  As  part  of  our 
review  of  availability,  we  will  separate  maintainability  into  its  components 
(preventive  and  corrective  maintenance  aud  administrative  and  logistics  delay 
times)  to  determine  the  impact  of  these  individual  elements  on  overall  system 
availability. 


Availability  is  defined  as  a  measure  of  the  degree  to  which  an  item  is  in  an 
operable  and  commitable  state  at  the  start  of  a  mission  when  the  mission  is 
called  for  at  a  random  point  in  time. 

ELEMENTS  OF  AVAILABILITY 

As  is  evident  by  its  very  nature,  approaches  to  availability  are  time-related. 
Figure  4-1  illustrates  the  breakdown  of  total  equipment  time  into  those  time- 
based  elements  on  which  the  analysis  of  availability  is  based.  Note  that  the 
time  designated  ar  "off  time"  does  not  apply  to  availability  analyses  because 
during  this  time  system  operation  is  not  required.  Storage  and  transportation 
periods  are  examples  of  "off  time" . 


FIGURE  4-1.  BREAKDOWN  OF  TOTAL  EQUIPMENT  TIME 
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The  letters  "C"  and  "P"  represent  those  periods  of  time  attributed  to  cor¬ 
rective  or  preventive  maintenance,  respectively,  which  are  expended  in  active 
repair  of  hardware  or  in  waiting  (delay)  for  resources  to  effect  needed 
repairs.  This  waiting  or  delay  period  can  further  be  subdivided  into  admin¬ 
istrative  and  logistics  delay  periods. 


Definitions  of  commonly  used  availability  elements  are  given  below.  Several 
are  displayed  pictorially  in  Figure  4-1. 


TT  =  Total  intended  utilization  period,  total  time. 

TCM  =  Total  corrective  (unscheduled)  maintenance  time  per  specified 

period. 

TPM  =  Total  preventive  (scheduled)1  maintenance  time  per  specified 

period. 

ALDT  =  Administrative  and  logistics  down  time  spent  waiting  for  parts, 
administrative  processing,  maintenance  personnel,  or  transpor¬ 
tation  per  specified  period.  See  Figure  4-1,  Delay-Down  Time 
(no  maintenance  time) . 

TMT  =  Total  maintenance  time  =  TCM  +  TPM.  See  Figure  4-1,  Active-Down 
Time. 


TOT  =  Total  down  time  =  TMT  +  ALDT. 

(if  =  Operating  time  (equipment  in  use).  See  Figure  4-1. 

ST  =  Standby  time  (not  operating  but  assumed  operable)  in  a  specified 

period.  See  Figure  4-1. 

KT2F  =  Mean  time  between  failures. 


MTBM  -  Mean  time  between  maintenance  actions. 


MTBUMA  =  Mean  time  between  unscheduled  maintenance  actions  (unscheduled 
means  corrective) . 

MDT  =  Mean  down  time. 

MTTR  =  Mean  time  to  repair. 


The  basic  mathematical  definition  of  availability  is 


Availability  = 


Dp  Time  _ _ Up  Time _ 

Total  Time  “  Up  Time  +  Down  Time 


(4.1) 
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Actual  assessment  of  availability  is  accomplished  by  substituting  the  time- 
based  elements  defined  above  into  various  forms  of  this  basic'  equation. 
Different  combinations  of  elements  combine  to  formulate  different  definitions 
of  availability. 

Operational  availability  is  the  most  desirable  form  of  availability  to  be  used 
in  assessing  a  system's  combat  potential.  Achieved,  and  to  a  lesser  degree 
inherent  availability  are  primarily  the  concern  of  the  developing  agency  in 
its  interface  with  the  contractor  and  other  co-developing  agencies. 

Ao  is  an  important  measure  of  system  effectiveness  because  it  relates  system 
hardware,  support  and  environment  characteristics  into  one  meaningful 
parameter — a  figure  of  merit  depicting  the  equipment  state  at  the  start  of  a 
mission.  Because  it  is  an  effectiveness-related  index,  availability  is  used 
as  a  starting  point  for  nearly  all  effectiveness  and  force  sizing  analyses. 

Inherent  Availability  (Ai) 

Under  certain  conditions,  it  is  necessary  to  define  system  availability  with 
respect  only  to  operating  time  and  corrective  maintenance.  Availability 
defined  in  this  manner  is  called  inherent  availability  (Ai). 


Ai  = 


MTBF 


MTBF  +  MTTR 


(4.2) 


Under  these  idealized  conditions,  we  choose  to  ignore  standby  and  delay  times 
associated  with  scheduled  or  preventive  maintenance,  as  well  as  administrative 
and  logistics  down  time.  Because  only  corrective  maintenance  is  considered  in 
the  calculation,  the  MTBF  becomes  MTBUMA,  and,  likewise,  MTTR  is  calculated 
using  only  times  associated  with  corrective  maintenance. 

Inherent  availability  is  useful  in  determining  basic  system  operational  char¬ 
acteristics  under  conditions  which  might  include  testing  in  a  contractor's 
facility  or  other  controlled  test  environment.  Likewise,  inherent  avail¬ 
ability  becomes  a  useful  term  to  describe  combined  reliability  and  maintain¬ 
ability  characteristics  ox  to  define  one  in  terms  of  the  other  during  early 
conceptual  phases  of  a  program  when,  generally,  these  terms  cannot  be  defined 
individually.  Since  this  definition  of  availability  is  easily  measured,  it  is 
frequently  used  as  a  contract-specified  requirement. 


As  is  obvious  from  this  definition,  inherent  availability  provides  a  very  poor 
estimate  of  true  combat  potential  for  most  systems,  because  it  provides  no 
indication  of  the  time  required  to  obtain  required  field  support.  This  term 
should  normally  not  be  used  to  support  an  operational  assessment. 

Case  Study  No.  4-1  displays  the  usefulness  of  inherent  availability. 

Operational  Availability  (Ao) 

Operational  availability,  unlike  inherent  availability,  covers  all  segments  of 
time  that  the  equipment  is  intended  to  be  operational  (total  time  in 
Figure  4-1).  The  same  up-down  time  relationship  exists  but  has  been  expanded. 
Up  time  now  includes  operating  time  plus  nonoperating  (stand-by)  time  (when 
the  equipment  is  assumed  to  be  operable) .  Down  time  has  been  expanded  to 
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include  preventive  and  corrective  maintenance  and  associated  administrative 
and  logistics  lead  time.  All  are  measured  in  clock  time. 

OT  +  ST 

Operational  Availability  =  Ao  =  nT  +  +  A)Pf  *  (^-3) 

This  relationship  is  intended  to  provide  a  realistic  measure  of  equipment 
availability  when  the  equipment  is  deployed  and  functioning  in  a  combat  en¬ 
vironment.  Operational  availability  is  used  to  support  operational  testing 
assessment,  life  cycle  costing,  and  force  development  exercises. 

One  significant  problem  associated  with  determining  Ao  is  that  it  becomes 
costly  and  time-consuming  to  define  the  various  parameters.  Defining  ALDT  and 
TPM  under  combat  conditions  is  not  feasible  in  most  instances.  Nevertheless, 
the  operational  availability  expression  does  provide  an  accepted  technique  of 
relating  standard  reliability  and  maintainability  elements  into  an 
effectiveness-oriented  parameter.  As  such,  it  is  a  useful  assessment  tool. 

Case  Study  4-4  illustrates  how  this  relationship  can  be  used  to  define  and 
analyze  the  various  elements  of  reliability  and  maintainability.  Case  Study 
4-2  illustrates  the  calculation  of  Ao. 

One.  important  aspect  to  take  note  of  when  assessing  Ao  is  that  it  is  affected 
by  utilization  rate.  The  less'  a  system  is  operated  in  a  given  period,  the 
higher  'Ao  will  be.  It  is  important  therefore  when  defining  the  "total  time" 
period  to  exclude  lengthy  periods  during  which  little  or  no  system  usage  is 
anticipated.  Case  Study  4"3  attempts  to  display  this  characteristic  of  Ao. 

One  other  frequently  encountered  expression  for  operational  availability  is 


Ao  = 


MTBM 

MTBM  +  MDT  * 


(4.4) 


where 


MTBM  =  mean  time  between  maintenance  actions  and  MDT  =  mean  down  time. 

While  maintenance-oriented,  this  form  of  Ao  retains  consideration  of  the  same 
basic  elements.  The  MDT  interval  includes  corrective  and  preventive  mainte¬ 
nance  and  administrative  and  logistics  down  time.  This  form  of  the  Ao  rela¬ 
tionship  would  generally  prove  more  useful  in  support  of  early  maintenance 
parameter  sensitivity  and  definition  analysis.  Note  that  the  above  definition 
assumes  that  standby  time  is  zero . 

Achieved  Availability  (Aa) 


This  definition  of  availability  is  mathematically  expressed  as 


Aa  = 


OT 

OT  +  TCM  +  TPM  - 


(4.5) 


Aa  is  frequently  used  during  development  testing  and  initial  production  test¬ 
ing  when  the  system  is  not  operating  in  its  intended  support  environment. 
Excluded  are  operator  before-and-after  maintenance  checks  and  standby,  supply, 
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and  administrative  waiting  periods.  Aa  is  much  more  a  system  hardware- 
oriented  measure  than  is  operational  availability,  which  considers  operating 
environment  factors.  It  is,  however,  dependent  on  the  preventive  maintenance 
policy,  which  is  greatly  influenced  by  non-hardware  considerations. 


The  following  paragraphs  present  a  generalized  approach  for  evaluating  system 
availability.  It  is  important  to  note  that  for  such  an  analysis  to  be  mean¬ 
ingful  to  an  equipment  user  or  developer  it  must  reflect  the  peculiarities  of 
the  system  being  considered. 

General  Procedure 


1.  The  operational  and  maintenance  concepts  associated  with  system  utiliza¬ 
tion  must  be  defined  in  detail  using  terminology  compatible  with  the  user, 
developer  and  contractor. 

2.  Using  the  above  definitions,  construct  a  time  line  availability  model 
(see  Figure  4-2)  which  reflects  the  mission  availability  parameters. 


FIGURE  4-2  MISSION  AVAILABILITY  TIME  LINE  MODEL  GENERALIZED  FORMAT 
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NOTE:  Figure  4-2  displays  elements  of  availability  frequently  included  in  a 
quantitative  assessment  of  availability.  The  up  or  down  status  of  a 
■  specific  system  during  preventive  maintenance  must  be  closely  examined. 
Generally,  a  portion  of  the  preventive  maintenance  period  may  be  con¬ 
sidered  as  uptime.  Cold  standby  time  must  also  be  examined  closely 
before  determining  system  up  or  down  status  during  this  period. 

3.  With  the  aid  of  the  time  line  model,  determine  which  time  elements  rep¬ 
resent  ’'uptime"  and  "downtime."  Don't  be  mislead  by  the  apparent  simplicity 
of  this  task.  For  example,  consider  that  the  maintenance  concept  may  be 
defined  so  that  the  equipment  must  be  maintained  in  a  comuittable  state  during 
the  performance  of  preventive  maintenance. 


Additionally,  for  multi-mission  and/or  multi-mode  systems,  it  will  be  neces¬ 
sary  to  determine  up  and  down  times  as  a  function  of  each  mission/mode.  This 


generally  will  require  the  use  of  a  separate  time  line  model  for  each  identi¬ 
fiable  miss ion/ mode. 


Likewise,  separate  time  line  models  are  generally  required  to  support  the 
availability  analyses  of  systems  which  experience  significantly  different 
peacetime,  sustained  combat  and  surge  utilization  rates. 

4.  Determine  quantitative  values  for  the  individual  time  elements  of  the 
'time  line  models.  Coordinate  these  values  with  the  user,  developer  and 

contractor . 

5.  Compute  and  track  availability  using  the  definitions  of  availability 
appropriate  for  the  current  stage  of  system  development. 

6.  Continue  to  check  availability  model  status  and  update  the  model  as 
required.  Special  attention  should  be  given  to  updating  the  model  as  the 
operational,  maintenance,  and  logistics  support  concepts  mature. 

System  Availability  Assessment  Considerations 

As  indicated  in  the  above  paragraphs,  the  quantitative  evaluation  of  avail¬ 
ability  must  be  carefully  and  accurately  tailored  to  each  system.  For  this 
reason,  no  detailed  examples  are  presented  in  this  text.  However,  the  follow¬ 
ing  paragraphs  do  present  concepts  which  will  apply  to  various  classes  of 
systems . 

Recovery  Time.  Normally,  availability  measures  imply  that  every  hour  has 
equal  value  from  the  standpoint  of  operations  and  the  performance  of  mainte¬ 
nance  and  logistics  activities.  Normally,  the  operational  concept  requires 
the  system  to  function  only  for  selected  periods ,  The  remaining  time  is 
traditionally  referred  to  as  "off-time,"  during  which  no  activity  is 
conducted . 

An  alternative  to  the  "off-time"  or  "cold,  standby"  concepts  is  the  use  of  the 
term  "recovery  time"  (RT) . 

FIGURE  4-3  MISSION  AVAILABILITY  TIME  LINE  MODEL  RECOVERY  TIME  FORMAT 
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Recovery  time  represents  an  interval  of  time  during  which  the  system  may  be  up 
or  down.  Recovery  time  does  not  appear  in  the  availability  calculation  which 


is  based  only  on  the  TT  time  period.  Take  special  note  of  the  fact  that  cor¬ 
rective  maintenance  time  (TCM)  is  found  in  both  TT  and  RT  time  intervals. 
Corrective  maintenance  performed  during  the  TT  period  is  maintenance  required 
to  keep  the  system  in  a  mission  ready  or  available  status.  Corrective  mainte¬ 
nance  performed  during  the  RT  period  generally  addresses  hardware  malfunctions 
which  do  not  result  in  a  non  mission-ready  status. 

The  principal  advantage  of  using  the  "recovery  time"  analysis  is  that  it  can 
provide  a  more  meaningful  availability  assessment  for  systems  whose  periods  of 
required  availability  are  predictable  and  whose  preventive  maintenance  consti¬ 
tutes  a  significant  but  delayable  portion  of  the  maintenance  burden. 

The  recovery  time  calculation  technique  concentrates  the  availability  calcula¬ 
tion  during  the  operational  time  period,  thereby  focusing  attention  on 
critical  up  and  down  time  elements. 

The  above  discussion  presents  an  alternate  technique  of  computing  system 
availability,  i.e.,  the  use  of  the  recovery  time  concept.  Whatever  technique 
is  selected  for  computing  availability,  it  must  be  carefully  tailored  to  the 
system  undergoing  assessment 

Definition  of  the  terms  used  in  availability  analysis  must  be  stressed.  For 
example,  what  total  time  (TT)  period  has  been  chosen  for  an  analysis  base? 
Assume  for  a  moment  that  we  are  assessing  the  Ao  of  an  operational  squadron 
and  that  we  have  chosen  a  7-day  TT  period.  If  the  aircraft  normally  are  not 
flown  on  weekends  or  are  left  in  an  up  condition  on  Friday  night  it  is  obvious 
that  Ao  will  be  higher  than  if  a  5-day  total  time  were  selected.  Reference 
the  discussion  of  recovery  and  standby  time.  See  Case  Study  4-3. 

Other  definitions  associated  with  Ao  are  not  quite  so  obvious  and  must  be 
included  in  pretest  definition.  For  example,  are  "before  and  after"  opera¬ 
tional  checks  conducted  in  conjunction  with  preventive  maintenance  excluded 
from  down,  time  because  the  equipment  is  assumed  operable?  Similarly,  are 
corrective  maintenance  diagnostic  procedures  logged  against  down  time?  What 
if  the  hardware  is  not  found  defective?  How  is  ALDT  arrived  at?  Is  it  as¬ 
sumed,  calculated  or  observed?  What  is  ths  operational  status  of  a  system 
during  the  warm  standby  period? 


The  expression  for  availability  frequently  provides  the  vehicle  needed  to 
analyze  other  system  requirements  both  directly  and  by  way  of  trade-offs. 
Case  Studies  4-4  and  4-5  provide  examples  of  this  application. 


For  many  modern  weapon  systems,  availability  is  not  simply  an  "up"  or  "down" 
condition.  Systems  such  as  AEGIS  and  HAWK  have  multi-mission/mode  capabili¬ 
ties  and  thus  require  detailed  techniques  to  characterize  the  associated 
availability  states.  While  these  multi-mission/mode  characterizations  may 
appear  different,  they  are  indeed  based  on  the  expressions  presented  pre¬ 
viously.  The  definition  of  terms,  modes  and  states  is  equally  important  in 
the  analysis  of  these  complex  systems.  The  reliability  of  a  multi-mission 
system  is  examined  in  Case  Study  2-7. 


There  are  a  number  of  computer  simulation  models  available  which  are  well 
suited  for  evaluating  interactions  between  system  design  characteristics, 
logistic  support,  and  relevant  operational  output  measures  such  as  operational 
availability  or  sortie  rate.  Examples  of  such  models  include  LCOM  (aircraft), 
CASEE  and  PRISM  (carrier-based  aircraft) ,  ARMS  (Army  aircraft) ,  TIGER  (Ship 
systems),  RETCOM  (combat  vehicles),  etc.  These  models  provide  visibility  of 
manpower  and  test  equipment,  queueing  effects,  and  the  impact  of  spares  stock- 
age  levels  on  operational  availability,  which  generally  cannot  be  evaluated 
with  simple  analytical  formulas.  Simulation  models  are  particularly  useful 
for  using  test  results  tc  project  operational  availability  under  conditions 
different  from  the  test  environment  (e.g.,  to  project  availability  under 
wartime  surge  conditions).  One  drawback  to  simulation  models  is  that  they  are 
usually  more  cumbersome  to  use  than  straightforward  analytical  techniques. 


CASE  STUDY  NO.  4-1 


4 ta*,: 


Background 

Early  in  the  development  phase  of  a  new  avionics  system,  it  is  determined  that 
an  inherent  availability  of  0.92  is  required.  The  reliability  and  maintenance 
engineering  personnel  in  the  program  office  desire  to  analyze  only  what  effect 
this  requirement  has  on  the  relationship  between  their  disciplines,  which  is 
appropriate  in  a  first-look  consideration. 

Determine 


How  can  this  analysis  be  accomplished? 

Solution 

A  straightforward  analysis  can  be  conducted  by  using  the  definition  of  Ai. 
Remember  Ai  does  not  consider  delay  times  nor  preventive  maintenance.  Should 
the  engineers  so  desire  and  if  it  is  considered  important  for  this  system, 
they  could  redefine  MTTR  to  include  all  maintenance. 

Ai  = _ MTBF__  =  o  92 

MTBF  +  MTTR 

MTBF  =  (0.92) (MTBF  +  MTTR) 

MTBF  =  (11. 5) (MTTR)  or 
MTTR  =  (0.09) (MTBF) 

The  function  MTTR  =  (0.09)(M!BF),  may  be  used  directly,  or  it  may  be  plotted 
as  shown  below.  The  graph  is  a  straight  line,  passing  through  the  origin, 
with  a  slope  of  0.09.  For  the  same  form  of  equation,  the  general  solution  is 
MTTR  =  [(1-A)/A]  (MTBF),  where  A  is  inherent  availability. 


A  PLOT  OF; 
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.j. 


MTTR 


CASE  STUDY  NO.  4-2 


Background 

A  system  has  an  MTTR  of  30  minutes  and  an  MTB'UMA  of  50  hours.  The  intended 
utilization  of  the  system  is  5,000  hours  per  year.  Ten  hours  of  preventive 
maintenance  are  required  for  each  1,000  hours  of  operation.  A  mean  admin¬ 
istrative  and  logistic  delay  time  of  approximately  20  hours  is  associated  with 
each  unscheduled  maintenance  action. 


Determine 


For  a  one-year  period,  determine  OT,  TCM,  TPM,  ALBT,  ST,  Ao,  Aa,  and  Ai  for  a 
utilization  of  5,000  hours  per  year.  Determine  Ao  if  MTTR  were  reduced  to 
zero.  Determine  the  maximum  number  of  operation  hours  in  a  year.  Compare  Ao 
and  Ai. 


Solution 


TT  =  (365) (24)  =  8,760  hours 
OT  =  5,000  hours 

TCM  =  (0.5)  =  50  hours 


TPM 

ALDT 

ST 

Ao 


Aa 


Ai 


ifSbo  (5’00°)  =  50  hours 

(20)  =  2,000  hours 

8,760  -  (5,000  +  50  +  50  +  2,000)  =  8,760  -  7,100  -  1,660 

5,000  +  1,660  n  ,, 

87760  -  0>76 


5,000 

5,000  +50+50 


0.98 


5,000 

5,000  +  50  " 


0.99 


If  MTTR  (ior  corrective  maintenance  only)  were  reduced  to  essentially  zero 

Ao  =  . 5,000  +  (1,660+50) _ =  6,710 

5,000  +  (1,660+50)  +  50+0+2,000  8,760 


Ao  -  0.77 

NOTB:  50  hours  added  to  numerator  represents  additional  available  standby 
time.  This  time  had  been  spent  on  repair  when  MTTR  was  non-zero. 


Assuming  Sm  =  Q,  the  maximum  possible  operating  hours  in  a  year  can  be  deter¬ 
mined  as  follows: 


Ao  = 


5^000 


5,000  +  50  +  50  +  2,000 


0.704 


(Ao) (hours/year)  “  (0.704)(8,760)  =  6,153  hours  maximal®. 


An  alternative  method  for  determining  maximum  possible  operating  hours  assum¬ 
ing  ST  =  0  is  to  solve  the  following  equation  for  x. 


8760  -  ~~  x  +  y— ~  x  +  x 
50  1000  50 


-  x, 


where 


0.5 

50 

10 

1000 

20 

50 


x  =  TCM 

x  =  TPM 

x  =  ALDT. 


The  solution  is  x  «  6,169.  The  difference  in  the  two  values  occurs  as  a 
consequence  of  rounding  Ao  (0.704)  to  three  significant  digits. 


! 


’awmiuma.'.w.--. .... . 


CASE  STUDY  NO.  4-3 


Background 

Test  planning  requires  that  an  assessment  be  made  of  some  basic  program  re¬ 
quirements.  During  this  assessment  v  you  observe  that  some  of  the  assumptions 
concerning  availability  assessment  are  questionable.  Consider  the  case  where 
system  availability  is  being  computed,  and  let  us  assume  that  we  have  the 
option  of  using  either  a  5-day  test  period  or  a  7-day  test  period.  Note  that 
neither  system  utilization  nor  maintenance  occurs  on  2  of  the  7  days.  A  close 
review  of  these  conditional  is  warranted,  particularly  one  which  permits  the 

week  for  total,  time  when  in  fact  additional  system 


utilization 

of  a  7-d 

usage  does  nop  occur  < 

Determine 

What  is  the  impact  of 

For  purposes 

of  this 

OT  = 

10  hours 

TPM  » 

5  hours 

TCM  ■ 

60  hours 

ALDT  a 

22  hours 

Solution 

For: 

7  Days 

Ao  - 


OT  +  ST 


OT  I-  ST  +  TPM  +  TCM  +  ALDT 


OT  =  10  hours 

ST  -  158  hours 

A  "  1&8  +  87 
Ao  a  0 . 66 


5  Days 

OT  -  10  hours 

ST  a  110  hours 


Ao  - 


120 


120  +  87 
Ao  =  O'. 58 


Commentary 


Note  the  higher  value  obtained  by  including  the  two  additional  non-usage  days. 


CASE  STUDY  NO.  4-4 


Solving  for  the  MTBF,  we  obtain 


MTBF 


(DTF) (OT) _ 

(l-Ao)Crr)  -  TPM  • 


(5) 


Using  the  following  values 

XT  =  90  days  x  24  hrs/day  =  2,160  hours 

OT  =  23  missions  x  40  hours  per  mission  =  920  hours 

DTF  =  24  hours  per  failure 

TPM  =  100  hours 

and  substituting  into  (5) ,  we  obtain 

MTBF  =  TF678)(2[l60^  -  100  =  50,0  huurs* 

NOTE:  When  using  this  definition  of  MTBF,  it  is  important  to  verify  that  the 
standby  time  is  not  forced  below  a  zero  value  by  erroneously  defining 
OT,  TDT,  and  TT. 


CASE  STUDY  NO.  4-5 


Background 

A  system  currently  in  late  development  has  exhibited  an  operational  availabil¬ 
ity  of  0.66.  The  user  has  stipulated  that  an  operational  availability  of  a 
0.75  minimum  is  required.  In  order  to  improve  system  operational  availabil¬ 
ity,  it  is  decided  to  change  the  maintenance  concept  on  several  low- 
reliability  units.  The  current  circuit  card  remove/ replace  concept  will  be 
changed  to  a  black  box  remove/ replace.  The  following  tabulations  list  the 
characteristics  of  the  existing  equipment  and  those  desired  after  the  system 
maintenance  concept  has  been  revised  in  an  attempt  to  improve  operational 
availability. 


Existing  Elements 

TT  -  168  hours 
TPM  =  5  hours 

TCh  =  60  hours 
ALDT  =  22  hours 
Ao  -  0.66 

Determine 


Desired  Elements 

TT  =  16b  hours 
TPM  *  5  hours 

TCh  =  to  be  determined 
ALDT  =  22  hours 
Ao  --  0.75 


New  required  value  of  TCM  which  must  be  realized  if  the  desired  Ao  increase  is 
to  be  achieved. 

Solution 

.  _  OT  +  ST  _  TT 

"  OT  +  ST  +  TCM  +  TPM  +  ALDT  “  TT  +  TCM  +  TPM  +  ALDT 

i 

An  -  168  -  168 

“168+5  +  TCM  +  22  ”  TCM  +  195 
Since  Ao  -  0.75, 

TCM  a  27.4  hours 
Commentary 

Of  course,  the  reasonableness  or  attainability  of  such  a  reduction  must  be 
considered.  Increased  operational  availability  also  can  be  obtained  by  de¬ 
creasing  TPM  or  ALDT. 
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CHAPTER  5 


'  • _  MATHEMATICAL  MODELS  OF  RELIABILITY 

INTRODUCTION 

In  a  general  sense,  we  cans  say  that  the  science  of  statistics  attempts  to 
quantify  our  uncertainty  about  the  outcome  of  an  experiment  or  test.  Before 
we  conduct  an  experiment,  we  afev  aware  of  the  possible  outcomes,  but  we  are 
not  sure  which  of  the  outcomes  will  actually  result.  The  mathematical  model, 
a  logical  extension  of  our  assumptions  about  the  experiment,  quantifies  our 
uncertainty  and  provides  us  with  a  tool  for  analysis  of  the  outcome. 

Suppose  that  a  system  is  to  be  put  into  operation.  We  know  that  after  a 
certain  amount  of  operating  time  it  will  experience  a  failure.  We  would  like 
to  know  when  the  system  will  fail.  Generally,  any  prediction  about  the  actual 
time  of  failure  will  not  be  accurate  enough  to  be  worthwhile.  However,  we  ctn 
address  more  confidently  questions  such  as:  "Will  the  system  operate  free  of 
failure  for  at  least  500  hours?"  A  model  that  describes  the  experiment  in 
question  will  help  us  answer  these  questions  with  a  certain  amount  of 
assurance . 

In  this  chapter,  we  consider  three  models  that  are  pertinent  to  RAM  consid- 
erations:  the  binomial,  the  Poisson,  and  the  exponential.  The  binomial  and 
Poisson  are  discrete  models  in  that  they  essentially  count  numbers  of  fail¬ 
ures.  The  exponential  model  is  continuous  in  that  it  describes  times  between 
failures.  Although  the  Poisson  mode)  addresses  discrete  events,  it  is  a 
continuous  time  model.  It  counts  failures  as  they  occur  over  a  period  of 
time,  i.e.,  the  possible  outcomes  of  the  experiment,  conducted  on  a  continuous 
time  basis,  are  enumerated  as  numbers  of  failures.  This  distinction  will 
become  clearer  as  we  contrast  the  Poisson  model  and  the  binomial  model. 

In  its  most  basic  form,  a  mathematical  model  of  a  statistical  experiment  is  a 
mathematical  expression  (function)  that  defines  the  probability  associated 
with  each  of  the  outcomes  of  the  experiment.  For  our  purposes,  we  will  dis¬ 
cuss  the  two  basic  types  of  models:  discrete  and  continuous.  The  type  of 
model--discrete  or  continuous--is  defined  by  the  type  of  outcome  that  the 
experiment  provides. 

ihSlCMIe  models 

A  discrete  model  is  appropriate  when  the  possible  outcomes  of  an  experiment 
can  be  enumerated  or  counted.  In  its  most  basic  form,  the  discrete  model  is  a 
mathematical  expression  (function)  that  defines  the  probability  of  each  indi¬ 
vidual  outcome.  A  simple  example  is  the  following.  Suppose  a  die  is  to  be 
tossed  once,  and  the  outcome  of  interest  is  the  number  of  dots  facing  up  when 
the  die  comes  to  rest.  The  outcomes  are  {1,  2,  3,  4,  5,  6}.  If  we  assume 
that  the  die  is  "fair,"  the  probabilities  can  be  expressed  as  follows: 

p(l)  =  p(2)  =  p(3)  =  p(4)  =  p(5)  =  p(6)  =  1/6. 

Graphically,  we  display  the  probabilities  in  Figure  5-1. 
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A  statistical  experiment  often  results  in  an  outcome  that  is  measured  on  a 
continuous  scale.  Time  and  distance  are  perhaps  the  most  common  continuous 
variables.  In  its  most  basic  form,  the  continuous  model  is  a  mathematical 
expression  (function)  useful  in  computing  probabilities  of  certain  outcomes. 
It  differs  from  probabilities  for  discrete  models  in  that  it  does  not  define 
probabilities  directly.  Generally,  for  a  continuous  model,  it  only  makes 
sense  to  consider  the  probability  of  an  outcome  within  a  range  of  values  or  a 
certain  interval--between  100  and  150  miles,  more  than  10  hours.  The  prob¬ 
ability  of  an  outcome  falling  within  a  given  range  is  the  area  which  lies 
beneath  the  continuous  model  curve  over  that  range.  Consider  the  examples 
below. 


FIGURE  5-3  GRAPH  OF  THE  EXPONENTIAL  MODEL 


The  probability  that  an  outcome  is  between  1  and  2  is  defined  by  the  area 
under  the  curve  between  the  values  1  and  2.  Therefore, 

P(outcome  falls  between  1  and  2) 


“X 

e  dx  =  -e 


-x 


2 

1 


e"1  -  e"2  =  0.233. 


The  probability  that  an  outcome  is  between  0.50  and  0.51  is 


-x 

e 


dx 


-0.50 


which  is  e 


-0.51 


or  0.006. 


FIGURE  5-4  GRAPH  OF  UNIFORM  MOOEL 


Figure  5-4  illustrates  another  possible  continuous  model  with  the  function 
being  f(x)  =0.1  defined  on  the  interval  10  <  x  <  20. 

The  probability  that  an  outcome  is  less  than  13  is 

r13 

f  0.1  dx, 

J  10 

which  ic  1.3- 1.0,  or  0.3.  The  probability  that  an  outcome  is  between  16  and 
16.1  is 


r  16.1 

k  0..1  dx, 

J  16 

which  is  1.61- 1.60,  or  0.01. 

WiLU 

The  model  that  is  used  most  commonly  to  describe  the  outcomes  of  success/fail 
test  programs  is  the  binomial  model.  In  order  for  a  testing  program  to  be  a 
binomial  experiment,  four  conditions  are  required.  They  are: 

-  The  test  period  ''onsists  of  a  certain  number  (n)  of  identical  trials. 

-  At  any  individual  time  uait  (trial),  the  ter.  results  in  a  success  or 
failure. 

-  The  outcome  at  any  individual  time  unit  is  independent  of  the  outcomes 
of  all  other  time  units. 

-  The  reliability  (probability  of  success)  of  the  system  remains  un¬ 
changed  for  each  trial. 
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Ia  a  lot  acceptance  sampling  test  program,  the  third  and  fourth  conditions  are 
not  precisely  satisfied.  In  such  a  case,  the  hypergeometric  distribution 
provides  the  exact  analysis  and  must  be  used  for  small  lot  sizes.  However,  if 
the  sample  size  (n)  is  small  compared  to  the  lot  size  (N) ,  say  n/N  <  0.05, 
then  the  bincmial  model  provides  a  reasonable  basis  for  analysis.  We  do  not 
present  a  discussion  of  the  hypergeometric  distribution  in  this  text.  The 
reader  can  refer  to  any  number  of  moderate  level  statistics  textbooks  for 
information  on  the  use  of  hypergeometric  distribution. 


Examples  of  binomial  experiments  are: 


firing  a  missile  (is  it  launched  successfully?); 


-  firing  a  missile  (does  it  hit  the  intended  target?); 


-  operating  any  system  over  time  (does  it  achieve  its  durable  life?). 


For  the  remainder  of  this  section  on  binomial  models,  the  following  notation 
is  used. 


n:  the  number  of  trials  or  test  units. 


p:  the  probability  of  failure  for  any  trial.  (We  use  p  here  as  the 
probability  of  failure  instead  of  the  more  classical  probability  of 
success  because  of  the  failure-oriented  approach  used  by  the  Poisson 
model) . 


/n\  binomial  coefficient,  which  by  definition  is  equal  to  n! / [k! (n-k) ! ] , 
\k/'  where  k  must  be  some  integer  between  0  and  n  inclusive.  By  defini- 


where  k  must  be  some  integer  between  0  and  n  inclusive.  By  defini¬ 
tion,  n!=n(n-l)(n-2) . . . 1  and  0 ! =1 . 


b  (k):  the  probability  of  k  failures  out  of  n  trials  with  p  the  probability 


of  failure  on  any  one  trial. 


B  (k) :  the  probability  of  k  or  fewer  failures  out  of  n  trials  with  p  the 
probability  of  failure  on  any  one  trial. 


Any  binomial  experiment  (test  program)  is  completely  characterized  by  the 
number  of  trials  and  the  probability  of  failure  for  any  given  trial.  The 
probability  of  failure  (p)  is  generally  an  unknown  value  about  which  testing 
requirements  are  stated. 


In  Chapter  7,  the  binomial  is  used  in  this  reversed  role:  the  test  results 
will  have  been  observed  and  we  will  make  inferences  about  the  probability  of 
success  based  on  the  binomial  model. 


For  the  binomial  model  an  exact  mathematical  formula  is  used  to  compute  prob¬ 
abilities.  We  denote  the  probability  of  exactly  k  failures  out  of  n  trials 

for  a  fixed  probability  of  failure,  p,  by  b  (k),  and 

n ,  p 


VP«  ■  (2)  >k«-P>' 


(5.1) 


mk 


The  probability  of  k  or  fever  failures,  termed  the  binomial  cumulative  distri¬ 
bution  function,  is  B  00,  where 

»,P 

■  j0  (“) 

k 

=  lb  (i).  (5.2) 

i— 0  Q,p 

Figure  5-5  presents  graphs  of  three  binomial  models.  The  graphs  chosen  por¬ 
tray  models  for  which  the  values  of  p  are  equal  to  0.2,  0.5,  and  0.7  with  a, 
the  number  of  trials,  eq^al  to  10  in  each  case. 

For  the  case  p  =  0.2,  the  graph  shows  that  a  large  number  of  failures--more 
than  5 — is  unlikely.  Note  that  the  most  likely  number  of  failures  is  2,  which 
corresponds  to  a  percentage  of  failures  equal  to  0.2.  For  the  case  p  =  0.7, 
the  graph  shows  that  a  small  number  of  failures — fewer  than  5 — is  unlikely. 
Once  again,  the  most  likely  number  of  failures,  7,  corresponds  to  a  percentage 
of  failures  equal  to  0.7.  For  the  case  p  =  0.5,  the  graph  shows  that  a  mod¬ 
erate  number  of  failures--between  3  and  7--is  likely,  with  5  failures  (half  of 
the  trials)  being  most  likely. 

Computation  of  binomial  probabilities  using  bQ  or  Bq  ^  is  cumbersome  when  n 

is  large.  Three  alternative  methods  for  determining  binomial  probabilities 
are: 


-  use  of  a  statisticrii  package  for  a  calculator, 

-  use  of  binomial  tables,  and 

-  use  of  an  approximating  distribution  (Poisson  or  normal). 

Tables  of  b  _  or  B  for  n  less  than  or  equal  to  20  are  generally  published 
n,p  n»P 

in  elementary  statistic?  textbooks.  More  extensive  tables  are  available  but 
are  not  as  easy  to  locate.  A  table  of  cumulative  binomial  probabilities  for 
selected  values  of  r.,  k,  and  p  is  given  in  Appendix  B,  Table  1.  When  n  is 
larger  than  20,  either  the  Poisson  distribution  (p  >  0.8  or  p  <  0.2)  or  the 
normal  distribution  (0.2  <  p  <  0.8)  provide  reasonable  approximations  to 
binomial  probabilities.  (See  Appendix  A  for  details  on  these  procedures.) 

In  Case  Studies  5-1,  5-2  and  5-3,  we  demonstrate  the  application  of  the 
binomial  model  and  compute  probabilities  associated  with  the  model. 
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POISSON  MODEL 


The  most  commonly  used  model  for  describing  the  outcomes  of  a  continuous  time 
testing  program  is  the  Poisson  model.  In  order  for  a  testing  program  to  be  a 
Poisson  experiment,  it  is  required  that  no  more  than  one  event  (failure)  can 
occur  at  the  same  time  and  that  the  number  of  events  (failures)  is  related 
directly  to  the  amount  of  test  time.  Examples  of  Poisson  experiments  are: 

-  number  of  failures  of  a  system  during  a  test  cycle; 

-  number  of  unscheduled  maintenance  actions  required  during  a  given  time 
period;  and, 

-  number  of  misfires  of  an  automatic  weapon  firing  over  a  given  time 
period. 

For  the  remainder  of  the  section  on  Poisson  models  and  the  succeeding  section 
on  exponential  models,  the  following  notation  is  used. 

X:  Failure  rate  or  average  number  of  failures  per  unit  time. 


T:  The  length  of  the  interval  (hours,  miles,  etc.)  of  interest 
(e.g.,  mission  duration,  test  exposure). 

T(k):  The  probability  of  exactly  k  failures  during  a  test  period  of 
*  length  T  when  the  failure  rate  is  X. 

G\  Tlie  Probability  of  k  or  fewer  failures  during  a  test  period  of 

’  length  T  when  the  failure  rate  is  X. 

Any  Poisson  experiment  (test  program)  is  completely  characterized  by  the 
length  of  time  on  test  and  the  mean  value  function.  The  mean  value  function 
for  a  specific  test  length  is  the  average  number  of  failures  to  occur  during 
the  specified  length.  When  the  system  has  a  constant  failure  rate  for  the 
entire  interval  T,  this  function  is  simply  XT.  A  discussion  of  constant 
failure  rate  assumptions  can  be  found  in  Chapter  7.  The  value  X  is  the  fail¬ 
ure  rate  of  the  system  on  test.  A  more  familiar  parameter  is  the  mean  time 
between  failures  (MTBF)  which  is  the  reciprocal  of  X.  System  requirements  are 
generally  stated  in  terms  of  the  mean  time  between  failures. 

As  with  the  binomial  model,  Poisson  probabilities  can  be  computed  using  an 
exact  mathematical  formula.  We  denote  the  probability  of  exactly  k  failures 
during  a  test  period  of  length  T  where  the  failure  rate  is  X,  by  g^  ^,(k),  and 


*X,T 


/1l  m^k  -XT 
(it)  -  s^.Ti_e — 
w  k! 


(5.3) 


The  number  of  failures  may  be  any  integer  value  including  0  and  0!  =1.  The 
probability  of  k  or  fewer  failures,  termed  the  Poisson  cumulative  distribution 


r iiift  Trudi  wm  ii»m 


.,V. 
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function,  is  T(k),  where 


G>  T(k)  =  2 


X,T 


i=0 


i! 


2-  g\  m  (i)  • 
i=0  *»T 


(5.4) 


Figure  5-6  presents  graphs  of  three  Poisson  models.  Values  for  XT  of  2,  5, 
and  7  were  chosen  to  demonstrate  the  effect  of  time  T  on  the  numbers  of  fail¬ 
ures  likely  to  be  seen.  When  X,  the  failure  rate,  is  fixed,  the  number  of 
failures  will,  in  all  likelihood,  increase  with  the  amount  of  operating  time 
T. 


Alternative  methods  for  computing  Poisson  probabilities  include: 

-  use  of  a  statistics  package  for  a  calculator, 

-  use  of  Poisson  tables  or  charts,  and 

-  use  of  an  approximating  function. 

In  Case  Studies  5-5  and  5-6  we  demonstrate  the  application  of  the  Poisson 
model  and  compute  probabilities  associated  with  the  model.  Tables  of  g^  T(k) 

or  G^  T(k)  are  available  in  many  textbooks.  A  table  of  cumulative  Poisson 

probabilities  is  given  in  Appendix  B,  Table  3.  Appendix  B,  Chart  1  is  also 
useful  in  determining  cumulative  Poisson  probabilities.  When  the  product  XT 
is  greater  than  5,  the  normal  distribution  provides  reasonable  approximations 
to  Poisson  probabilities.  (See  Appendix  A  for  details  on  this  procedure.) 

WMm*  MODEL 

Generally,  it  is  more  informative  to  study  times  between  failures,  rather  than 
numbers  of  failures,  for  a  continuous  time  testing  program.  The  most  commonly 
used  model  for  describing  the  times  between  failures  for  a  continuous  time 
testing  program  is  the  exponential  model.  In  order  for  a  testing  program  to 
qualify  as  an  exponential  experiment,  the  following  conditions  are  required: 
(1)  the  system  is  as  good  as  new  after  each  repair,  and  (2)  the  probability  of 
failure  in  any  given  interval  of  time  is  the  same  no  matter  how  old  a  system 
is  and  no  matter  how  many  failures  it  has  experienced.  The  second  condition 
is  an  intuitive  description  of  the  so-called  memoryless  property.  The  system 
cannot  "remember"  how  old  it  is,  nor  can  it  "remember"  how  often  it  has 
failed.  The  examples  listed  for  Poisson  experiments  can  serve  also  as 
examples  of  exponential  experiments. 

For  the  exponential  model,  there  is  an  exact  mathematical  formula  used  for 
computing  probabilities  that  a  certain  amount  of  time  will  pass  before  the 
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next  failure.  The  probability  that  a  failure  will  occur  iu  some  future  inter¬ 
val  of  time  (a,  b)  for  a  system  with  failure  rate  X  is 


FIGURE  5-7  GENERAL  EXPONENTIAL  MODEL  i  f  (x)*Xe~ 


OUTCOME 


The  exponential  cumulative  distribution  function,  F^(t) ,  defines  the  prob¬ 
ability  that  the  system  will  fail  before  time  t.  By  definition, 


A  function  of  more  interest  to  us  is  the  reliability  function,  R(t),  which 
defines  the  probability  that  the  system  will  operate  for  t  units  of  time 
without  failure.  By  definition, 


(5.7) 


-  ( t/Q'i 

The  reliability  function,  R(t),  can  also  be  expressed  as  e  '  ,  where  0,  the 
reciprocal  of  X,  is  the  mean  time  between  failures  (MTBF).  The  reliability 
function  R(t)  translates  the  effectiveness  parameters  X,  the  failure  rste,  or 
0,  the  MTBF,  into  reliability.  Reliability  is  the  probability  of  failure-free 
operation  for  a  specified  length  of  time,  t. 


We  referred  to  the  variable  t  as  a  time  variable  in  the  above  discussion. 
Measure  of  life  units  which  can  be  appropriate  are  hours,  miles,  cycles  and 
rounds . 


As  an  example,  suppose  that  the  mission  profile  for  a  system  requires  a 
mission  duration  (MD)  of  40  hours  and  the  system  has  a  mean  time  between 
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operational  mission  failure  (MTBOMF)  of  400  hours.  Then  the  probability  that 
the  system  successfully  completes  a  mission  can  be  evaluated  using  the  reli¬ 
ability  function,  R(t) .  How 

R(t)  =  e"Xt  =  e’t/6  =  e-(MD/*fTBOMF) 

Since  MD  =  40  hours  and  MTBOMF  =  400  hours,  the  mission  reliability  is 

e  ^0/^00^  yjjich  reduces  to  0.905.  In  other  words,  the  system  has  a  90.5% 
chance  of  completing  the  40-hour  mission. 

NOTE:  A  useful  approximation  for  the  reliability  function  is: 

-t/0  *  1  -  t/Q  for  t/0  <  0.1. 

For  the  above  example,  t/0  =  0.1  so  that  the  approximation  yields  0.90. 

In  Case  Study  5-4,  we  demonstrate  the  application  of  the  exponential  model 
with  computations  based  on  both  models.  See  Case  Studies  5-4,  5-5,  and  5-6, 
for  more  illustrations  of  this  computation  and  other  computations  associated 
with  the  Poisson/ exponential  model. 


-At 

The  reliability  function  e  may  also  be  sym¬ 
bolised  as  exp(-At)  or  In  *(~At).  That  is,  it 
is  the  exponential  function  evaluated  at  the 
point,  -At,  or  it  is  the  inverse  of  the  natural 
logarithm  function  evaluated  at  that  point. 
Some  calculators  evaluate  the  exponential  func¬ 
tion  as  the  inverse  natural  logarithm  function. 


CASE  STUDY  NO.  5-1 

Background 

A  sensor  device  has  an  operational  mission  reliability  of  0.90  for  a  4  hour 
mission.  At  least  3  sensors  are  required  to  locate  targets.  Failed  sensors 
will  not  be  repaired  during  a  mission. 

Determine 

1.  If  a  system  employs  3  sensors,  what  is  the  probability  of  successfully 
completing  the  mission? 

2.  If  a  system  employs  4  sensors,  what  is  the  probability  of  successfully 
completing  the  mission? 

Solution 

1.  We  use  the  mathematical  formula  for  the  binomial  model  given  in  equation 
5.1,  with 

R  =  1  -  p  =  0.9 
p  =  1  -  0.9  =  0.1 
n  =  3,  and 
k  -  0 

Applying  equation  5.1,  the  probability  of  0  failures  is: 

b3,oa(0)  *  (oK1^0-9)3 
The  binomial  coefficient  is: 


The  probability  of  0  failures  is: 

(1) (1) (0 . 9 ) 3  =  0.729. 


5-13 


2.  We  use  the  binomial  cumulative  distribution  function  given  in  equation 
5.2,  with 


R  a  1  -  p  =  0.9 
p  -  1  -  0.9  =  0.1 
n  -  4,  and 
k  =  1 

Applying  equation  5.2,  the  probability  cf  1  or  fewer  failures  is: 

B4,0.l‘»  “  j0(t)(O  J)k<O-9)<4'U 

=  (q)(0.1)°(0.9)4  +  (4)(0.1)1(0.9)3 
The  binomial  coefficients  are: 

(S)  -  bt4t  -  »•  —  (t)  -  iftr  -  *• 

The  probability  of  1  or  fewer  failures  is: 

(1)(1)(0.9)4  +  (4)(0.1)1(0.9)3 
=  0.656  +  0.292  =  0.948. 

Commentary 

For  the  second  problem,  we  use  the  binomial  cumulative  distribution  function 
since  we  are  required  to  compute  the  probability  of  0  failures  or  1  failure, 
i.e.,  the  cumulative  probabilities  of  both  outcomes. 


'< !  flit 


CASE  STUDY  NO.  5-2 


A  lot  of  500,000  rounds  of  ammunition  is  available  for  sale.  The  buyer  will, 
purchase  the  lot  if  be  has  reasonable  assurance  that  the  lot  is  no  more  than  * 
15%  defective .  > 

\ 

Determine 


1.  If  the  true  proportion  of  defects  is  0.15,  what  is  the  probability  a 
sample  of  size  10  will  yield  fewer  than  2  defects?  More  than  3  defects? 


2.  If  the  -true  proportion  of  defects  is  0.05,  what  is  the  probability  a 
sample  of  size  20  will  yield  fewer  than  3  defects?  More  than  1  defect? 


3.  If  the  true  proportion  of  defects  is  0.02,  what  is  the  probability  a 
sample  of  size  100  will  yield  fewer  than  5  defects? 

4.  If  the  true  proportion  of  defects  is  0.15,  what  is  the  probability  a 
sample  of  size  50  will  yield  more  than  10  defects?  Between  5  and  10  defects, 
inclusive? 


Solutions 


This  is  a  lot  acceptance  sampling  problem.  Although  the  binomial  model  is  not 
technically  correct,  it  will  provide  very  good  approximations  in  this  case 
because  the  ratio  of  sample  size  n  to  lot  size  N(N  =  500,000)  is  not  more  than 
0.0002.  A  further  explanation  of  the  use  of  the  binomial  model  for  lot  ac¬ 
ceptance  sampling  problems  is  found  on  page  5-5. 

1.  The  probability  of  failure,  p,  is  0.15.  For  fewer  than  2  defects,  we 
look  in  the  tables  for  n  =  10,  the  column  p  =  0.15,  and  the  row  c  =  1.  The 
probability  is  0.5443. 

The  probability  of  more  than  3  defects  is  the  difference  between  1  and  the 
probability  of  fewer  than  4  defects.  The  probability  of  fewer  than  4  defects 
is  0.9500,  so  the  probability  of  more  than  3  defects  is  0.05. 

2.  The  probability  of  fewer  than  3  defects  out  of  20  is  obtained  directly  in 
the  tables  for  n  =  20,  the  column  p  =  0.05,  and  the  row  c  =  2.  The  prob¬ 
ability  is  0.9245. 

The  probability  of  more  than  1  defect  is  the  difference  between  1  and  the 
probability  of  fewer  than  2  defects.  The  probability  of  fewer  than  2  defects 
is  0.7358,  so  the  probability  of  more  than  1  defect  is  0.2642. 

3.  A  binomial  table  for  n  =  100  is  not  given  in  the  Appendix  and  would  be 
difficult  to  locate  in  other  sources.  However,  because  the  sample  size,  n,  is 
large,  we  may  use  an  approximation  fairly  confidently.  Recall  that  there  are 
two  approximations  (Poisson  and  normal)  to  the  binomial  presented  in  the  text 
of  Chapter  5.  The  procedures  for  using  these  approximations  are  detailed  in 


Appendices  A-l  and  A-2.  Using  the  normal  approximation  (Appendix  A-l),  the 
probability  of  4  or  fewer  defects  out  of  100  trials  is 


1  -  P(Z  >  (4  +  0.5  -  (100) (0. 02) )/Vl00 (0.02) (0.98))  , 
which  reduces  to 

1  -  P(Z  >  1.79)  =  0.9633  . 

Using  the  Poisson  approximation  (Appendix  A-2),  we  set  m  =  2,  c  =  4,  and 
obtain  the  probability  directly  from  Appendix  B,  Table  3  as  0.947. 

The  exact  value  for  the  probability  of  fewer  than  5  defects,  obtained  from  the 
formula,  is  0.9491.  Note  that,  although  each  approximation  is  reasonably 
close  to  the  exact  value,  the  Poisson  has  provided  the  better  approximation. 
As  noted  on  page  5-6,  the  Poisson  is  more  appropriate  when  p  is  very  large  or 
very  small  (in  our  case,  p  =  0.02). 

4.  The  probability  of  more  than  10  defects  is  the  d'  Tference  between  1  and 
the  probability  of  fewer  than  11  defects.  The  probability  of  fewer  than  11 
defects  using  the  binomial  tables  for  n  =  50,  the  column  p  =  0.15,  and  the  row 
c  =  10,  is  0.8801.  The  probability  of  more  than  10  defects  is  thus  0.1199. 
The  probability  of  between  5  and  10  defects  inclusive  is  the  probability  of 
fewer  than  11  defects  less  the  probability  of  fewer  than  5  defects.  These  two 
numbers  are  0.8801  and  0.1121,  and  the  difference  is  0.7680.  The  normal 
distribution  is  appropriate  for  approximating  this  probability.  Using  this 
approximation,  we  find  that  the  probability  of  more  than  10  (11  or  more) 
defects  is 

P(Z  >  (11  -  0.5  -  (50) (0. 15))/V50(0. 15) (0.85))  , 
which  reduces  to 


P(Z  >  1.19)  =  0.1170  . 

The  approximate  probability  of  between  5  and  10  defects  inclusive  is 
P(Z  >  (5  -  0.5  -  7.5)/V6 .38)  -  P(Z  >  (10  +  0.5  -  7.5)A/6T38) 

=  P(Z  >  -1.19)  -  P(Z  >  1.19)  =  0.7660  . 


Commentary 


We  have  calculated  probabilities  for  certain  outcomes  which  could  result  from 
an  inspection  of  a  sample  from  the  lot  of  500,000.  To  calculate  these  values 
it  is  necessary  to  assume  that  we  know  the  true  proportion  of  defects  in  the 
entire  lot.  Of  couse,  we  do  not,  in  fact,  know  this  true  proportion  of  de¬ 
fects,  but  we  perform  this  exercise  in  order  to  develop  a  rational  plan  for 
sampling  from  the  lot  in  order  to  determine  whether  we  should  accept  or  reject 
the  lot.  Consider,  for  example,  the  solution  to  the  second  part  of  ques¬ 
tion  1.  Namely,  the  probability  of  4  or  more  defects  out  of  10  trials  when 
the  lot  contains  15%  defects  is  0.05.  Consequently,  if  the  sample  of  10 
should  yield  4  or  more  defects,  the  buyer  has  reasonable  assurance  that  the 
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lot  contains  wore  than  15%  defects  and  should  be  rejected.  Consider  now 
part  2  of  question  4.  In  a  preliminary  step  to  the  solution,  we  determined 
that  the  probability  of  4  or  fewer  failures  when  the  lot  contains  15%  defects 
is  0.1121.  Consequently,  if  a  sample  of  50  yields  4  or  fewer  defects,  the 
buyer  has  reasonable  assurance  that  the  lot  contains  fewer  than  15%  defects 
and  should  be  accepted.  We  discuss  in  Chapter  8  methods  for  preparing  test 
plans  which  provide  reasonable  assurance  to  bath  producer  and  consumer.  Our 
purpose  in  presenting  the  above  discussion  is  to  introduce  the  reader  to  the 
probabilistic  analysis  that  takes  place  in  the  preparation  of  test  plans  or 
lot  acceptance  sampling  plans. 
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CASE  STUDY  NO.  5-3 


Background 

A  *•-  hanized  infantry  battalion  has  36  armored  personnel  carriers  (APC) ,  20 
w  stations  (WS) ,  and  5  command  posts  (CP) .  The  mission  reliability  of  an 

APi,  .s  0.70,  of  a  WS  is  0.85,  and  of  a  CP  is  0.fJ5.  The  entire  battalion  can 
perform  satisfactorily  if  at  least  24  APCs,  17  WSs,  and  5  CPs  are  operable 
throughout  the  mis*  >n. 

Keteran'* 

1.  Wnat  is  the  probability  eacn  type  of  system  has  enough  ui  its  operating  to 
complete  the  mission? 

2.  What  1&  the  probability  the  mission  will  be  successful? 

3.  How  small  do  we  need  to  make  p,  the  probability  of  failure,  in  order  to 
ensure  that  each  set  of  systems  has  a  probability  0.90  of  performing 
satisfactorily? 

4.  If  ead*  of  the  probabilities  of  failure  is  fixed,  then  how  many  more  of 
each  type  of  system  is  required  to  achieve  the  goal  mentioned  in  number 
3? 

Solutions 


The  probability  that  a  sufficient  number  of  APCs  operate  throughout  the  mis¬ 
sion  is  the  probahilitv  of  12  or  fewer  failures  out  of  36  trials.  Note  that 
p,  the  probability  of  failure,  is  equal  to  0.30.  The  probability  that  a 
sufficient  number  of  WSs  operate  throughout  the  mission  is  the  probability 
that  2  nr  fever  failures  occur  out  of  20  trials.  The  probability  that  all  CPs 

operate  throughout  the  mission  is  che  probability  of  no  failures. 

l.a.  The  probability  of  i2  or  fewer  failures  out  of  36  trials  where  p  =  0.30. 

i.  Uae  Appendix  B,  Table  1.  Oa  page  B-20  for  n  =  36,  we  look  in  the 

column  labeled  0.300  and  the  row  r  =  12.  The  value  is  0..  7365. 

ii.  Use  Normal  Approximation.  (See  Appendix  A-l.)  Since  n.p  =  (36) 

(0.3)  -  10.8  and  np(l-p)  =  (36)(0.3)(0.7)  =  7.56,  the  approximate 
probability  is 

1  -  P(Z  >  (12.5-np)/VopTT"P)) 

=  1  -  F(Z  >  (12. 5-10. 8) 7^775^7 


We  obtain  this  value  using  Appendix  B,  Table  2,  page  B-42  in  the  row 

marked  z_  *  0.62  and  the  column  labeled  P(Z  >  2  ). 

a  -  a 

iii.  Use  Poisson  Approximation.  (See  Appendix  A-2.)  We  make  the  identi¬ 
fication  a  =  ap  =  (36) (0.3)  =  10.8  and  use  Appendix  B,  Table  3.  On 
page  B-47  we  obtain  the  value  from  the  column  labeled  12  and  inter¬ 
polate  between  rows  labeled  10.5  and  11.0.  The  value  is  0.718. 

Note  that  since  0.2  <  p  <  0.8,  the  normal  yields  a  better  approximation 
than  the  Poisson. 

b.  The  probability  of  3  or  fewer  failures  out  of  20  trials  where  p  =  0.15. 

i.  Use  Appendix  B,  Table  1.  On  page  B-7  for  n  =  20,  we  look  in  the 

column  labeled  0.150  and  the  row  r  =  3.  The  value  is  0.6477. 

ii.  Use  Normal  Approximation.  (See  Appendix  A-l.)  Since  np  =  (20) 
(0.25)  =  3  and  np(l-p)  =  (20) (0. 15)(0.85)  =  2.55,  the  approximate 
probability  is 

1  -  P(Z  >  (3.5-np)/Vnp(l-p)) 

=  1  -  P(Z  >  (3.5-3)/V2755) 

=  1  -  P(Z  >  0.32)  =  1  -  0.3745 

a  0.6255 

We  obtain  this  value  using  Appendix  B,  Table  2,  page  B-42  in  the  row 
marked  zQ  «=  0.32  and  the  column  labeled  P(Z  >  z^). 

iii.  Use  Poisson  Approximation.  (See  Appendix  A-2.)  We  make  the  identi¬ 
fication  ■  =  np  =  (20) (0.15)  -  3  and  use  Appendix  B,  Table  3.  On 
page  B-46  we  obtain  the  value  from  the  column  labeled  3  and  the  row 
labeled  3.00.  The  value  ic  0.647. 

Note  that  since  p  <  0.2,  the  Poisson  yields  a  better  approximation  than 
the  normal. 


c.  The  probability  of  0  failures  out  of  5  trials  where  p  =  0.05.  This 
probability  is  given  by  equation  5.1. 


.n-k 


Vr00  *  u)  pka'p)' 

(jj)  (0.05)°(0.95)5  ~  (0 . 95 ) 5  =  0.774 


Note  the  value  of  n  is  much  too  small  to  use  an  approximation.  See 
discussion  of  this  concept  on  page  5-6. 
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t 

1 


Author- 
System  ized 


Minimum 

Required 


Max.  #  of  System 
Allowable  Mission 
Failures  Reliability 


Probability 
of  Mission  Success 

Binomial  Normal  Poisson 


APC 

36 

24 

12 

WS 

20 

17 

3 

CP 

5 

5 

0 

0.70 

0.7365 

0.7324 

0.718 

0.85 

0.6477 

0.6255 

0.647 

0.95 

0.774 

- 

- 

The  term  System  Mission  Reliability  is  the  probability  that  an  individual  APC, 
WS,  or  CP  will  successfully  complete  a  mission.  The  term  Probability  of 
Mission  Success  is  the  probability  that  for  the  individual  systems  at  least 
the  requi red  minimum  number  of  vehicles  will  successfully  complete  the 
mission. 


2. 


Mission  Success 


We  use  a  series  model  to  evaluate  the  probability  of  mission  success.  This 
probability  is  the  product  of  the  probabilities  of  success  for  APC’s  (0.737), 
WS's  (0.648),  and  CP's  (0.774).  The  product  is  0.37. 

(See  Chapter  2's  section  entitled  "Series  and  Redundancy  Models.") 

3.  In  part  1  above,  we  determined  that  the  mission  success  probabilities  for 
APC's,  WS's,  and  CP's,  are  approximately  0.73,  0.65,  and  0.77,  respectively. 
If  we  assume  that  the  authorized  number  of  units  of  each  type  system  remains 
fixed,  then  we  must  improve  the  mission  reliability  of  each  type  system  to 
achieve  a  success  probability  of  0.90  for  each  type  system.  Generally, 
statistics  books  do  not  address  procedures  for  solving  this  type  problem  but 
the  solutions  are  straightforward  and  logical. 

a.  For  APC's,  the  probability  of  12  or  fewer  failures  must  be  at  least 
0.90. 


i.  Use  Appendix  B,  Table  1.  Note  that  for  n  =  36  and  c  =  12,  the 
values  in  the  body  of  the  table  increase  as  p,  the  probability 
of  failure  decreases.  This  fact  is  actually  true  for  any 
values  of  u  and  c.  As  we  move  in  the  body  of  the  table  from 
right  to  left,  we  find  that  the  first  time  the  probability 
exceeds  0.90  is  for  a  p  of  0.25.  Consequently  a  p  of  0.25  or 
less  will  achieve  the  goal. 

ii.  The  normal  approximation  provides  a  method  for  determining  the 
desired  probability,  p.  The  approximate  probability  of  12  or 
fewer  failures  out  of  36  trails  for  any  p  is 

1  -  P(Z  >  (12.5  -  36p)/V36p(lrpT)  • 
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The  procedure  is  to  set  the  above  quantity  equal  to  0.90  and 
solve  for  p.  To  accomplish  this,  note  that  we  can  reduce  the 
equation  to 

P(Z  >  (12.5  -  36p)A/36p(l-p))  =0.10  . 

Since  P(Z  >  1.28)  =  0.10,  the  above  equality  occurs  when 
(12.5  -  36p)A/36p(l-p)  =  1.28  . 

To  solve  the  equation  for  p,  we  multiply  both  sides  by 

V36p(l“P) 

and  square  both  sides  to  obtain  the  quadratic  equation 

,  156.25  -  900p  +  1296p2  =  58.98p(l-p),  or 

1354. 98pz  -  958. 98p  +  156.25  =  0  . 

We  used  the  quadratic  formula  to  find  p  =  0.254. 

iii.  The  Poisson  approximation  provides  a  method  for  determining  the 
desired  probability,  p.  Note  that  for  c  =  12  in  the  Poisson 
tables,  the  probabilities  increase  as  the  value  of  m  decreases. 
Searching  the  body  of  the  table  in  the  column  labeled  12, 
we  find  that  the  probability  exceeds  0.90  for  the  first  at 

h  =  8.6.  Recall  that  m  =  np  for  the  Poisson  approximation. 
Now  n  =  36  and  m  =  8.6,  so  p  =  8.6/36  =  0.24. 

For  WS's,  the  probability  of  3  or  fewer  failures  must  be  at  least 
0.90.  The  procedures  for  determining  the  probability  of  faiure,  p, 
are  identical  to  those  in  part  a  above. 

i.  Appendix  B,  Table  1.  For  c  =  3,  as  p  decreases,  the  value  in 
the  table  exceeds  0.90  for  the  first  time  at  p  =  0.09. 

ii.  Normal  Approximation.  The  equation  to  solve  is 

(3.5  -  20p)/V20p(l-p)  =  1.28  , 

and  the  y*lue  of  p  that  solves  the  equation  is  p  =  0.C92. 

iii.  Poisson  Approximation.  For  c  =  3,  as  m  decreases ,  the  value  in 
the  table  exceeds  0.90  for  the  first  time  at  m  =  1.7.  Conse¬ 
quently,  p  =1.7/20  =  0.085. 

For  CP's,  the  probability  of  0  failures  must  be  at  least  0.90. 
Using  a  direct  computation,  we  solve  the  following  equation  for  p: 


Equivalently, 


(1-p)5  =  0.90  or  1-p  =  (0.90)0,2  . 

The  solution  to  the  above  equation  is  a  p  of  0.02. 

4.  If  the  mission  reliability  of  the  different  systems  cannot  be  improved, 
then  in  order  tc  achieve  a  success  probability  of  0.90  for  each  system,  the 
number  of  authorized  units  must  be  increased.  This  allows  for  more  units  to 
fail  during  the  mission,  while  maintaining  the  required  strength  to  achieve 
mission  success. 

a.  For  APC's,  the  probability  of  at  least  24  units  performing  success¬ 
fully  must  be  at  least  0,90. 

i.  Appendix  B,  Table  1. 

n  =  36  P ( 12  or  fewer  failures)  =  0.7365 

n  =  37  PC 13  or  fewer  failures)  =  0.8071 

n  =  38  P(14  or  fewer  failures)  =  0.8631 

n  =  39  P C 15  or  fewer  failures)  =  0.9056 

The  requirement  is  satisfied  for  an  n  of  39 . 

ii.  Normal  Approximation.  Since  the  number  of  failures  allowed 
increases  by  one  for  every  increase  in  allocation  while  the 
number  of  successes  remains  constant  at  34,  we  can  formulate 
the  solution  to  this  question  more  easily  in  terms  of  suc¬ 
cesses.  The  approximate  probability  of  24  or  more  successes 
out  of  n  trials,  for  any  n,  when  p  =  0.7  is 

P(Z  >  (23.5  -  0.7n)/V072lTO  . 

We  set  the  above  term  equal  to  0.90  and  solve  for  n.  Since  P(Z 
>  -1.28)  =  0.90,  the  value  of  n  required  is  the  one  which 
solves  the  equation 

(23.5  -  0.7n)/VoT21n  =  -1.28  . 

We  perform  the  same  manipulations  as  in  part  3  above  to  obtain 
a  quadratic  equation  in  n,  one  of  whose  solutions  is  n  =  38.8. 

iii.  Poisson  Approximation.  Let  us  use  Chart  6  to  approximate  the 
solution.  The  abscissa  of  the  graph  is  labeled  0/T  which  is 
the  reciprocal  of  m  =  np.  Consider  the  points  where  the  curves 
cross  the  0.90  ordinate  line. 
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c 

§ZS 

n  =  (0.30/T)"1 

//  Successes 

12 

0.12 

27.8  £  (28) 

16  or  more 

13 

0.108 

30.9  £  (31)  . 

18  or  more 

14 

0.097 

34.4  £  (35) 

21  or  more 

15 

0.09 

37.0  £  (38) 

23  or  more 

16 

0.083 

40.2  £  (41) 

25  or  more 

This  method  shows  that  38  units  are  not  enough  and  that  41 
units  are  too  many.  Either  39  or  40  units  are  appropriate. 

b.  For  WS's,  the  probability  of  at  least  17  units  performing  success¬ 
fully  must  be  at  least  0.90. 


i.  Appendix  B,  Table  1. 


n  =  20  P (3  or 

n  =  21  P(4  or 

. .  =  22  P(5  or 


fewer  failures)  =  A. 6477 
fewer  failures)  =  0.8025 
fewer  failures)  ~  0.9001 


The  requirement  is  satisfied  for  an  n  of  22. 


c. 


ii.  Normal  Approximation.  The  equation  to  solve  is 

(16.5  -  0.85n)A/0.1275n  =  -1.28  . 

The  solution  is  n  =  21.9. 

iii.  Poisson  Approximation.  Use  Chart  6. 


c 

6/T 

n  =  (0.156/T)"1 

#  Successes 

4 

0.42 

15.9  (16) 

12 

5 

0.33 

20.2  (21) 

16 

6 

0.26 

25.6  (26) 

20 

Clearly,  this  method  indicates  that  either  22  or  23  should  be 
sufficient. 

For  CP's,  the  probability  of  at  least  5  units  performing  satis¬ 
factorily  must  be  at  least  0.90.  The  number  of  units  is  so  small 
that  approximations  are  not  appropriate.  We  can  solve  the  problem 
very  easily  using  the  formula  for  computation  of  binomial  probabil¬ 
ities  . 


■  (!) 


nl 

[k! (n-k) ! ) 


k,,  Nn-k 
P  (l“p) 
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CASE  STUDY  NO.  5-4 


Background 

A  new  combat  assault  vehicle  is  being  proposed.  The  mean  miles  between  mis¬ 
sion  failure  (MMBMF)  is  to  be  320  miles.  The  mean  miles  between  durability 
failures  (MMBDF)  is  to  be  5771  miles.  Historically,  both  mission  failures  and 
durability  failures  have  occurred  at  a  fairly  constant  rate. 

Determine 

1.  What  is  the  mission  reliability  for  a  50-mile  mission? 

2.  What  is  the  probability  of  operating  for  4000  miles  without  experiencing 
a  durability  failure? 

3.  What  MMBDF  is  required  to  have  an  80%  probability  of  operating  for  4000 
miles  without  a  failure? 

Solutions 

1.  We  use  the  reliability  function, 

R(t)  =  e"t/0, 

where  t  is  mission  duration  and  0  is  mission  MMBF.  Since  t  =  50  miles  and  0  = 
320  miles,  the  mission  reliability  is 


j-5°/32°  ,  0.655  _ 


2.  Durability  is  defined  as  the  probability  of  completing  4000  miles  of 
operation  without  suffering  a  power  train  durability  failure.  Again  we  use 
the  reliability  function, 

K(t)  =  e~t/0, 

where  t  is  the  durability  requirement  and  9  is  the  power  train  MMBDF.  Since  t 
=  40G0  miles  and  0  -  5771  miles,  the  mission  reliability  is 

e-*000/5771  .  0  50 

3.  Once  age.in  we  use  the  reliability  function, 

Kt)  «  e“^/G, 

but,  in  this  case,  i  -  4000  miles  and  P.(4000)  =  0.80.  By  solving  the  above 
equation  for  G,  we  have 


9  a 


-log  u(tT 
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Consequently, 


6  = 


4000 

-log  0.80 
e 

17,926  ailes  . 


Commentary 

Usually  durability  requirements  are  imposed  on  specific  subsystems  such  as  the 
power  train.  When  individual  subsystems  are  considered,  durability  failures 
are  likely  to  occur  with  an  increasing  failure  rate  due  to  physical  wearout  of 
components.  In  such  a  case,  the  use  of  the  exponential  model  would  not  be 
appropriate.  Other  distributional  models  of  life  length  (such  as  the  Weibull 
distribution)  would  be  appropriate  for  use  in  analyzing  durability. 


i 
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CASE  STUDY  NO.  5-5 


Background 

A  heavy  truck  uses  10  road  wheels.  Each  tire's  mean  miles  between  failures 
(MMBF)  is  10,000  miles.  Assume  a  series  model  for  the  10  tire  system. 

Determine 

1.  What  is  the  failure  rate  of  each  tire. 

2.  What  is  the  probability  that  a  single  tire  will  experience  no  failure 

during  a  1000  mile  mission? 

3.  Assuming  the  tires  operate  independently,  what  is  the  probability  of  no 
tire  failures  in  a  500  mile  mission? 

4.  What  is  the  MMBF  of  the  entire  system  of  10  tires. 

5.  How  many  spares  should  be  kept  so  that  a  500  mile  mission  can  be  com¬ 

pleted  at  least  90%  of  the  time?  99%  of  the  time? 

Solutions 

1.  The  failure  rate  (X)  is  the  reciprocal  of  the  mean  miles  between  failures 
(MMBF).  Consequently, 


X  = 


MMBF  10,000 


=  0.0001  . 


2.  Using  the  formula  for  computing  Poisson  probabilities  (equation  5.3)  with 
the  following  parameters 

Mission  Length  (T)  -  1000  miles 
Failure  Rate  (X)  =  0.0001, 


i  ! 


f 


ptf-' 


we  have 


g  (k)  = 
8X,rK;  k! 


=  e 


,^-XT 
“61 
-o.i 


P(no  failures)  =  =  a’"1  =  e^0*00^^ 


=  0.905 


This  value  could  have  been  obtained  using  Appendix  B,  Table  3,  using  m  =  XT  = 
0.1  and  c  =  0.  In  addition,  we  could  have  used  Appendix  B,  Chart  1  using  0/T 
=  1/XT  =  10  and  c  =  0. 


5-27 


3. 


For  a  single  tire  the  probability  of  no  failures  in  a  500  mile  mission  is 


(XT)ke~XT 

kf 


f (0.0001) (500) ]Qe~(°-0001)(50°) 
0! 


=  0.951  . 

Since  all  tires  are  in  series,  have  the  same  failure  rate  and  operate  inde¬ 
pendently,  the  probability  of  no  failures  among  all  10  tires  is 

CO . 951) 10  =  0.605  . 

4.  In  order  to  determine  the  MMBF  of  the  system  of  10  tires,  we  use  the 
reliability  function 

R(T)  =  e“T/6, 

where  T  is  mission  length  and  0  is  MMBF.  Solving  this  equation  for  0,  we 
obtain 


°  =  -loge  R(T)  • 

In  problem  3,  we  determined  that  R(500)  =  0.605  for  the  10-tire  system.  Thus, 

_ _ 500 

~  -l',ge  0.605 

=  1000  miles. 

5.  Mission  Length  (T)  =  500  miles. 

System  Failure  Rate  (X)  =  0.001. 

(Note:  MMBF  =  100Q  from  question  4) 

Using  Appendix  B.  Table  3  with  m  =  XT  =  0.5,  we  have 

P(no  failures)  =  0.607 

P(less  than  or  equal  to  1  failures)  =  0.910 

P(less  than  or  equal  to  2  failures)  =  0.986 

P(less  than  or  equal  to  3  failures)  =  0.998 

For  a  90%  chance  of  mission  completion,  one  spare  will  suffice.  For  a  99% 
chance  of  mission  completion,  3  spares  are  required.  However,  the  improvement 
in  reliability  froc  0.986  (?.  spares)  to  0.998  (3  spares)  does  not  appear  to 
warrant  the  extra  spare. 
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CASE  STUDY  NO.  5-6 


Background 

A  new  vehicle  is  under  development  which  is  required  to  have  a  MTBF  of  at 
least  200  hyurs  (A  no  more  than  0.005)  and  a  mean  time  to  repair  (MTTR)  of  no 
more  than  40  man-hours.  Assume  that  the  specifications  (200  hours  MTBF  and  40 
hours  MTTR)  have  been  met  by  contractor  A. 

Determine 


1.  What  is  the  probability  that  a  development  test  of  1000  hours  will  show 
at  least  10  failures? 

2.  What  is  the  probability  of  no  failures  during  the  first  400  hours  of  the 
test? 

3.  How  many  man-hours  of  repair  time  are  expected  during  the  1000-hour  test? 

4.  How  much  calendar  time  in  days — operating  and  maintenance — based  upon  an 
8-hour  day  should  be  programmed  so  that  at  least  10  total  repairs  will  be 
performed  by  a  team  of  4  mechanics  working  on  10  identical  systems?  We  desire 
that  there  be  at  least  10  failures  with  probability  0.90.  We  desire  to  take 
only  a  10%  risk  that  repair  time  allotted  will  be  insufficient.  Assume  that 
each  repair  takes  40  hours,  the  mean  repair  time.  Rework  the  problem  assuming 
each  repair  is  completed  within  60  hours. 

Solutions 


1.  The  background  to  this  study  indicates  that  this  is  a  continuous  time 
testing  situation.  In  addition,  we  are  interested  in  numbers  of  failures  and 
not  in  times  between  failures,  so  the  Poisson  model  is  used  for  this  analysis. 
Test  time  (T)  is  1000  hours  and  MTBF  is  200  hours  (A  =  0.005).  The  unknown 
value  is  the  probability  of  10  or  more  failures. 

Use  Appendix  B,  Table  3,  with  AT  =  (0.005) (1000)  =  5.0.  The  probability  of  10 
or  more  failures  =  1  -  P(9  or  fewer  failures). 

For  AT  =  m  =  5.0,  we  see  in  the  column  labeled  c  =  9  that  the  probability  of  9 
or  fewer  failures  is  0.968.  Consequently,  the  probability  of  10  or  more 
failures  is  0.032. 

2.  Test  time  (T)  is  400  hours  and  MTBI  is  200  hours  (A  =  0.005).  The  un¬ 
known  value  is  the  probability  of  0  failures. 

Use  Appendix  B,  Table  3,  with  AT  =  (0.005) (400)  =2.0. 

For  AT  =  m  =  2.0  we  see  in  the  column  labeled  c  =  0  that  the  probability  of 
zero  failures  is  0.135. 

3.  Test  time  (T)  is  1000  hours  and  MTBF  is  200  hours  (A  =  0.005).  MTTR  is 
40  hours  (A  =  0.025).  Each  repair  takes  40  hours  on  the  average.  There  are 


1000(0.005)  or  5  failures  expected.  The  expected  repair  time  is  5(40)  or  200 
man-hours . 

4.  Let  Tq  be  operating  time  needed  and  Tr  be  repair  time  needed.  We  desire 

that  T^  be  largo  enough  that  the  probability  of  seeing  10  or  more  failures  be 

at  least  0.9.  However  we  must  allow  repair  time  for  that  number  of  failures 
(n^)  which  represents  all  bat  the  upper  10%  of  the  distribution  of  numbers  of 

failures,  i.e.,  the  probability  that  more  than  n^  failures  occurring  is  0.10. 

a.  Determine  T  :  Probability  of  at  least  10  failures  must  be  at  least 

0.90.  0 

Note:  The  probability  of  10  or  more  failures  =  1  -  the  probability  of  9  or 
fewer  failures. 

The  probability  of  nine  or  fewer  failures  must  be  no  more  than  0.10.  Under 

the  column  labeled  c  =  9.0  of  Table  3,  we  find  by  interpolation  that  a  0.10 

probability  of  9  or  fewer  failures  is  achieved  when  m  =  14.2. 

Since  m  =  XT  and  m  =  14.2 

T  =  T  and 
o 

\  ~  0.005 

We  solve  the  following  equation  for  T^: 

Tq  =  m/\  =  14.2/0.005  =  2,840  hours. 

b.  Determine  n1Q:  Allow  for  19  failures  since  the  probability  of  20  or 
more  failures  is  just  under  0.10. 

c.  Determine  T^:  (19) (40)  =  760  man-hours 

T^  =  760/4  =  190  clock  hours. 

d.  Determine  number  of  days  needed: 

Operating  hours:  2,840 

Ten  systems  operating  hours:  284  for  each  system 
Maintenance  hoc.,.:!:  190 

Total  hours:  474 
Days  Required:  59.25 

e.  If  each  repair  takes  60  man-hours,  then  1140  man-hours  are  required. 
This  corresponds  to  285  clock  hours.  Consequently,  the  total  number 
of  hours  is  5C9  which  represents  71.1  working  days. 


Commentary 


For  number  4,  we  have  allowed  40  man-hours  for  each  repair.  If  the  average 
repair  time  is  indeea  40  man-hours,  then  760  total  man-hour j  should  be  a 
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reasonably  close  estimate.  Computation  of  the  risk  involved  with  this  ap¬ 
proach  is  complicated  and  beyond  the  scope  of  this  work.  One  procedure  to 
adopt  is  to  increase  the  allotted  time  per  repair  as  we  have  noted  in  4e. 
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CHAPTER  6 


STATISTICAL  CONCEPTS 


INTRODUCTION 

As  we  mentioned  in  Chapter  5,  our  assumptions  about  a  given  testing  .situation 
lead  us  to  the  choice  of  a  mathematical  model  to  characterize  the  reliability 
of  a  system.  However,  we  cannot  determine  the  actual  reliability  of  the  sys¬ 
tem  using  the  model  until  the  parameters  of  the  model,  p  for  the  binomial 
model  and  X  (or  6)  for  the  Pois3on  or  exponential  model,  have  been  specified. 
The  values  of  the  parameters  are  never  known  with  absolute  certainty.  As  a 
consequence,  some  form  of  sampling  or  testing  is  required  to  obtain  estimates 
for  these  parameters.  The  quality  of  the  estimates  is,  of  course,  directly 
related  to  the  quality  and  size  of  the  sample. 

POINT  ESTIMATES 

Point  estimates  represent  a  single  "best  guess"  about  model  parameters,  based 
on  the  sample  data.  A  distinguishing  symbol  commonly  is  used  to  designate  the 
estimate  of  a  parameter.  Most  commonly,  a  caret  or  "hat"  is  used  to  designate 
point  estimates  (e.g.,  §,  &(x) ,  X).  Quite  often,  and  for  our  purposes,  the 
caret  further  indicates  that  the  estimator  is  a  maximum  likelihood  estimator; 
that  is,  it  is  the  most  likely  value  of  the  parameter  of  the  model  which  is 
presumed  to  have  generated  the  actual  data. 

There  are  criteria  other  than  maximum  likelihood  used  for  a  single  "best 
guess."  One  other  is  unbiasedness.  For  an  estimator  to  be  unbiased,  we  mean 
that,  in  the  long  run,  it  will  have  no  tendency  toward  estimating  either  too 
high  or  too  low.  Tue  point  estimates  which  we  propose  for  p  in  the  binomial 
model  and  for  X  in  the  Poisson  and  exponential  models  are  both  maximum  likeli¬ 
hood  and  unbiased. 

CONFIDENCE  STATEMENTS 

Point  estimates  represent  a  single  "best  guess"  about  parameters,  based  on  a 
single  sample.  The  actual  computed  values  could  greatly  overestimate  or 
underestimate  the  true  reliability  parameters,  particularly  if  they  are  based 
on  a  small  amount  of  data.  As  an  example,  suppose  that  20  rounds  of  ammuni¬ 
tion  were  tested  and  18  fired  successfully. 

The  maximum  likelihood  and  unbiased  estimate  of  reliability  is  R  =  18/20  = 
0.9.  In  other  words,  the  system  most  likely  to  have  generated  18  successes  is 
one  whose  reliability  is  0.9.  Note  that  0.9  is  the  percentage  of  successes 
actually  observed  in  the  sample.  However,  a  system  whose  true  reliability  is 
somewhat  less  than  or  somewhat  more  than  0.9  could  reasonably  have  generated 
this  particular  data  set. 

We  use  confidence  limits  to  address  how  high  or  low  the  value  of  a  parameter 
could  reasonably  be.  A  90%  confidence  interval  for  reliability  is:  0.717  <  R 
<  0.982.  In  other  words,  if  being  reasonable  signifies  being  9C%  confident  of 
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being  right,  then  it  is  unreasonable  to  consider  that  a  system  whose  reli¬ 
ability  is  actually  less  than  0.717  or  one  whose  reliability  is  actually  more 
than  0.982  generated  the  18  successful  rounds.  When  we  desire  to  be  more 
confident,  say  95%  confident,  that  our  interval  contains  the  true  system 
reliability,  we  widen  our  interval,  i.e.,  we  expand  the  group  of  systems 
considered  to  have  reasonably  generated  the  data.  A  95%  confidence  interval 
tor  the  reliability  of  our  example  system  is:  0.683  <  R  <  0.988.  Since  we 
are  now  allowing  for  the  possibility  thct  the  system  reliability  could  be  a 
little  lower  than  0.717  —  namely,  as  low  as  0.683  —  or  a  little  higher  than 
0.982  —  namely,  as  high  as  0.988  —  we  can  now  afford  to  be  more  confident 
that  our  interval  indeed  contains  the  true  value.  For  a  fixed  amount  of 
testing,  we  CAn  only  increase  our  confidence  by  widening  the  interval  of 
reasonable  values. 


Suppose  that  we  desire  to  reduce  the  size  of  the  interval  while  maintaining 
the  same  level  of  confidence  or  to  increase  the  level  of  confidence  while 
maintaining  approximately  the  same  size  interval.  Either  of  these  objectives 
is  accomplished  through  increased  testing,  i.e.,  taking  a  larger  sample.  If 
the  system  test  had  resulted  in  27  successful  firings  out  of  30  attempts  (vice 
18  out  of  20),  the  point  estimate  is  still  0.9.  However,  the  90%  confidence 
interval  for  system  reliability  is:  0.761  <  R  <  0.972.  The  length  of  this 
interval  represents  a  20%  reduction  in  the  length  of  the  90%  confidence  inter¬ 
val  resulting  from  our  test  of  20  units.  The  95%  confidence  interval  for 
system  reliability  is:  0.734  <  R  <  0.979.  This  interval  represents  an  8% 
reduction  in  size,  but  our  confidence  has  increased  to  95%.  Figure  6-1  graph¬ 
ically  portrays  the  effect  on  interval  length  induced  by  changing  confidence 
levels  or  increasing  sample  size. 
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A  cautious,  conservative  person  who  buys  safe  investments,  wears  a  belt  and 
suspenders,  and  qualifies  his  statements  carefully  is  operating  on  a  high- 
confidence  level.  He  is  certain  he  won't  be  wrong  very  often.  If  he  is  wrong 
once  in  100  times,  he  is  operating  on  a  99%  confidence  level.  A  less  con¬ 
servative  person  who  takes  more  chances  will  be  wrong  more  often,  end  hence  he 
operates  on  a  lower  confidence  level.  If  he  is  wrong  once  in  20  times,  he  is 
operating  on  a  95%  confidence  level.  The  confidence  level,  therefore,  merely 
specifies  the  percentage  of  the  statements  that  a  person  expects  to  be  cor¬ 
rect.  If  the  experimenter  selects  a  confidence  level  that  is  too  high,  the 
test  program  will  be  prohibitively  expensive  before  any  very  precise  con¬ 
clusions  are  reached.  If  the  confidence  level  is  too  low,  precise  conclusions 
will  be  reached  easily,  but  these  conclusions  will  be  wrong  too  frequently, 
and,  in  turn,  too  expensive  if  a  large  quantity  of  the  item  is  made  on  the 
basis  of  erroneous  conclusions.  There  is  no  ready  answer  to  this  dilemma. 

We  can  interpret  confidence  statements  using  the  concept  of  risk.  With  a  90% 
confidence  statement,  there  is  a  10%  risk;  with  a  99%  confidence  statement, 
there  is  a  1%  risk.  Confidence  intervals  generally  are  constructed  so  that 
half  of  the  total  risk  is  associated  with  each  limit  or  extreme  of  the  inter¬ 
val.  Using  this  approach  with  a  90%  interval  for  reliability,  there  is  a  5% 
risk  that  the  true  reliability  is  below  the  lower  limit  and  also  a  5%  risk 
that  the  true  reliability  is  above  the  upper  limit.  We  can  therefore  state 
for  the  example  system  with  18  of  20  successes  that  we  are  95%  confident  that: 
R  >  0.717.  This  is  a  lower  confidence  limit  statement.  We  are  also  95% 
confident  that:  R  <  0.982.  This  is  an  upper  confidence  limit  statement.  See 
Figure  6-2. 
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The  classical  textbook  approach  to  confidence  intervals  has  been  to  specify 
the  desired  confidence  level  and  determine  the  limit  associated  with  this 
confidence  level.  This  approach  creates  a  twofold  problem.  First,  the  de¬ 
sired  confidence  level  has  to  be  determined.  Second,  the  limits  that  are 
generated  are  generally  not,  in  themselves,  values  of  direct  interest.  A  very 
practical  modification  ic  to  determine  the  level  of  confidence  associated  with 
a  predetermined  limit  value.  For  example,  the  minimum  value  of  a  reliability 
measure  that  is  acceptable  to  the  user  is  a  logical  lower  limit.  The  con¬ 
fidence  in  this  value  can  then  be  interpreted  as  the  assurance  that  the  user's 
needs  are  met.  See  Figure  6-3. 


6-3 


FIGURE  6-3  CONFIDENCE  INTERVALS  -  ACCE  PTABLE 
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The  confidence  level  for  a  lower  limit  of  0.8  is  8i%.  A  system  reliability  of 
0.8  is  the  user's  minimum  acceptable  value  (MAV) . 

HYPOTHESIS  TESTING 

While  confidence  limits  are  generally  used  to  define  the  uncertainty  of  a  pa¬ 
rameter  value,  an  alternative  approach  is  hypothesis  testing.  Both  approaches 
essentially  give  the  same  information.  Hypothesis  testing  can  be  used  to  dis¬ 
tinguish  between  two  values  or  two  sets  of  values  for  the  proportion  of  fail¬ 
ures  in  a  binomial  experiment,  or  for  the  failure  rate  in  a  Poisson/ 
exponential  experiment.  Let  us  examine  hypothesis  testing  using  a  binomial 
example.  Typically,  for  a  binomial  experiment,  it  is  hypothesized  that  the 
probability  of  failure,  p,  is  a  specified  value.  While  there  is  seldom  any 
belief  that  p  is  actually  equal  to  that  value,  there  are  values  of  p  which 
would  be  considered  unacceptable  in  a  development  program.  These  unacceptable 
values  are  specified  in  an  alternative  hypothesis.  Consider  the  following 
examples . 

(1)  One-Sided  Tests 

Hq ;  p  =  0.3  (Null  Hypothesis) 

Hj!  p  >  0.3  (Alternative  Hypothesis) 

In  Case  (1),  the  evaluator  hopes  that  p  is  no  more  than  0.3.  He  considers  a  p 
of  more  than  0.3  to  be  unacceptable.  This  is  a  classical  one-sided  test. 
Another  type  of  one-sided  test  has  the  alternative  hypothesis  p  <  0.3. 

(2)  Two-Sided  Tests 
Hq:  p  =  0.3 

Hj:  p  t  0.3  , 

In  Case  (2),  the  evaluator  hopes  that  p  is  approximately  0.3.  Values  of  p 
much  larger  than  or  much  smaller  than  0.3  are  unacceptable.  This  is  a  clas¬ 
sical  two-sided  test. 
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(3)  Simple  vs.  Simple  Tests 
Hq:  p  s  0.3 
Hx:  P  =  0.5 

In  Case  3,  the  evaluator  hopes  that  p  is  no  more  than  0.3.  He  considers  a  p 
of  more  than  0.5  to  be  unacceptable.  The  region  between  0.3  and  0.5  is  an 
indifference  region  in  that  it  represents  acceptable  but  not  hoped  for  values. 
This  is  actually  a  classical  simple  versus  simple  test.  This  type  of  test  is 
treated  extensively  and  exclusively  in  Chapter  8. 

In  order  to  conduct  a  statistical  test  of  hypothesis,  the  following  steps  are 
employed : 

1.  The  hypothesis,  null  and  alternative,  are  specified.  For  our  purposes, 
the  null  hypothesis  is  the  contractually  specified  value  (SV)  and  the 
alternative  hypothesis  is  the  minimum  acceptable  value  (MAV) . 

2.  A  sample  size,  n,  is  determined.  This  value  must  be  large  enough  to 
allow  us  to  distinguish  between  the  SV  and  MAV.  Chapter  8  is  devoted  to 
procedures  for  determining  a  sufficiently  large  value  of  n. 

3.  An  accept/reject  criterion  is  established.  For  our  purposes,  this 
criterion  is  established  by  specifying  a  value  c,  which  is  the  maximum 
number  of  failures  permitted  before  a  system  will  be  rejected. 

4.  The  sample  is  taken  and  the  hypothesis  is  chosen  based  upon  the  accept/ 
reject  criterion.  If  c  or  fewer  failures  occur,  we  accept  the  system. 
If  more  than  c  failures  occur,  we  reject  the  system. 

am  RISKS 

There  are  two  possible  errors  in  making  a  hypothesis-testing  decision.  We  can 
choose  the  alternative  hypothesis,  thereby  rejecting  the  null  hypothesis, 
when,  in  fact,  the  null  hypothesis  is  true.  The  chance  or  probability  of  this 
occurring  is  called  the  producer's  risk,  a.  On  the  other  hand,  we  can  choose 
the  null  hypothesis,  i.e.,  accept  it  as  reasonable,  when  in  fact  the  alter¬ 
native  hypothesis  is  true.  The  chance  or  probability  of  this  occurring  is 
termed  the  consumer's  risk,  0.  See  Chapter  8  for  an  additional  discussion  of 
this  topic. 

Consider  the  following:  A  system  is  under  development.  It  is  desired  that  it 
have  a  300-hour  MTBF.  However,  an  MTBF  of  less  than  150  hours  is  unaccept¬ 
able,  i.e.,  the  MAV  is  150  hours.  How  would  we  set  up  a  hypothesis  test  to 
determine  the  acceptability  of  this  new  system?  Our  null  hypothesis  (desired 
value)  is  that  the  MTBF  is  300  hours.  Our  alternative  hypothesis  (values  of 
interest)  is  that  the  MTBF  has  a  value  which  is  less  than  150  hours.  To 
decide  which  hypothesis  we  will  choose,  we  determine  a  test  exposure  and  a 
decision  criterion.  The  or  risk  (producer's  risk)  is  the  probability  that  the 
decision  criterion  will  lead  to  a  rejection  decision  when  in  fact  the  system 
meets  the  specification  of  300  hours  MTBF.  The  p  risk  (consumer's  risk)  is 
the  probability  that  the  decision  criterion  will  lead  to  an  acceptance  deci¬ 
sion  when  in  fact  the  system  falls  short  of  the  150  hours  MTBF. 
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For  a  given  test,  the  decision  criteria  can  be  altered  to  change  the  a  and  0 
risks.  Unfortunately,  a  decision  criterion  which  decreases  one  automatically 
increases  the  other.  The  only  way  to  decrease  both  risks  is  to  increase  the 
test  exposure,  that  is,  the  number  of  test  hours.  We  address  this  area  below 
in  Chapter  8,  "Reliability  Test  Planning." 

INTERFACE  BETWEEN  HYPOTHESIS  TESTING  AND  CONFIDENCE  STATEMENTS 

In  both  test  planning  and  data  analysis  situations,  either  hypothesis  testing 
or  confidence  statements  provide  an  avenue  of  approach.  The  interface  between 
the  two  approaches  can  be  best  understood  through  the  following  example. 

Suppose  a  is  the  desired  producer’s  risk  (a  =  0.05)  for  the  specified  MTBF  of 
300  hours.  Suppose  further  that  0  is  the  desired  consumer's  risk  (0  =  0.1) 
for  the  minimum  acceptable  MTBF  of  150  hours.  The  hypothesis  testing  approach 
determines  a  required  sample  size  and  a  specified  accept/reject  criterion.  We 
show  how  the  same  information  can  be  obtained  through  confidence  statements  in 
the  following  two  cases.  The  abbreviations  LCL  and  UCL  represent  Lower  Con¬ 
fidence  Limit  and  Upper  Confidence  Limit,  respectively. 

Note  that  the  distance  between  the  upper  and  lower  limits  is  the  same  as  the 
distance  between  the  SV  and  the  MAV.  When  this  is  the  case  we  shall  always  be 
able  to  make  a  clear-cut  decision  and  the  risks  associated  with  the  decision 
will  be  as  specified  at  the  outset  of  testing. 

F16URE  6-4  ACCEPTANCE  DECISION 
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Note  that  in  Figure  6-4  the  100(1-0)%  =  90%  lower  limit  exceeds  the  MAV  of  150 
hours.  In  addition,  the  100(l-a)%  =  95%  upper  limit  exceeds  the  specified 
value  of  300  hours.  The  consumer  is  90%  confident  that  the  150-hour  MAV  has 
been  met  or  exceeded  and  the  producer  has  demonstrated  that  the  system  could 
reasonably  have  a  30Q-ho.'r  MTBF.  Consequently,  we  would  make  the  decision  to 
accept  the  system. 


FIGURE  6-5  REJECTION  DECISION 
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Note  that  in  Figure  6-5  the  I00(l-p)%  =  90%  lower  limit  falls  below  the  MAV  of 
150  hours.  In  addition,  the  100(l-a)%  =  95%  upper  limit  falls  below  the  SV  of 
300  hours.  Therefore,  the  true  MTBF  could  reasonably  be  below  150  hours  and 
the  producer  has  not  demonstrated  that  an  MTBF  of  300  hours  is  reasonable. 
Consequently,  we  make  the  decision  to  reject  the  system. 


Perhaps  one  of  the  most  important  subjects  to  be  considered  in  the  evaluation 
of  RAM  characteristics  is  the  subject  of  test  exposure.  The  term  "test  ex¬ 
posure"  refers  to  the  amount  (quantity  and  quality)  of  testing  performed  on  a 
system  or  systems  in  an  effort  to  evaluate  performance  factors.  In  Chapter 
10,  we  discuss  the  qualitative  aspects  of  test  exposure  which  should  be  con¬ 
sidered  by  the  test  designer.  The  primary  purpose  of  Chapter  8,  "Reliability 
Test  Planning,"  is  to  document  procedures  which  ensure  that  the  quantitative 
aspects  of  test  planning  are  adequate. 

Recall  the  comment  we  made  in  the  previous  section  to  the  effect  that  the 
difference  in  the  distance  between  the  upper  and  lower  confidence  limits  was 
equal  to  the  difference  in  the  distance  between  the  SV  and  the  MAV.  When  this 
condition  is  achieved,  we  have  obtained  the  most  efficient  test  exposure  for 
the  stated  requirements  and  risks.  Examples  of  situations  where  test  exposure 
is  inadequate  or  excessive  are  given  below.  See  Case  Study  6-2  for  an  il¬ 
lustration  of  the  evaluation  of  a  proposed  test  exposure. 
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Note  that  in  Figure  6-6  the  100(l-p)%  =90%  lower  limit  falls  below  the  MAV  of 
150  hours.  The  100(l-ct)%  =  95%  upper  limit  exceeds  the  SV  of  300  hours.  The 
true  MTBF  could  reasonably  be  below  150  hours  or  above  300  hours.  Test  ex¬ 
posure  is  insufficient  to  discriminate  between  the  MAV  of  150  hours  and  the  SV 
of  300  hours  with  the  required  risk  levels  of  10%  and  5%.  If  we  reject  the 
system,  the  producer  can  legitimately  claim  that  an  MTBF  of  300  hours  is 
reasonable  for  his  system.  On  the  other  hand,  if  we  accept  the  system,  we  may 
be  fielding  an  inadequate  system. 


Note  that  in  Figure  6-7  the  100(1-$)%  =  90%  lower  limit  exceeds  the  MAV  of  150 
hours.  The  100(l-a)%  =  95%  upper  limit  falls  below  the  SV  of  300  hours.  The 
consumer  has  90%  confidence  that  the  150-hour  MAV  has  been  met  or  exceeded. 
However ,  the  producer  has  not  demonstrated  the  specified  300-hour  MTBF.  The 
test  exposure  is  more  than  required  to  obtain  the  risks  of  10%  and  5%  for  the 
stated  values  of  MAV  and  SV.  Since  the  MAV  has  been  met  or  exceeded,  we  will 
probably  accept  the  system.  We  may  have  paid  a  premium  to  obtain  information 
that  allowed  us  to  construct  a  confidence  interval  more  narrow  than  required. 

FIGURE  8-7  EXCESSIVE  TEST  DURATION 
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CASE  STUDY  NO.  6-1 


Background 

A  contract  for  a  new  electronic  system  specifies  an  MTBF  of  1000  hours.  The 
minimum  acceptable  value  is  500  hours  MTBF.  A  design  qualification  test  is  to 
be  conducted  prior  to  production.  The  test  risks  are  to  be  20%  for  consumer 
and  10%  for  producer. 

Determine 


Describe  the  events  which  lead  to  acceptance  or  rejection  of  the  system. 
Solution 


In  accordance  with  procedures  defined  in  Chapter  7,  "Reliability  Data 
Analysis,"  the  appropriate  hypothesis  test  is  set  up,  the  sample  is  taken,  and 
the  data  are  analyzed. 

The  Positive  Chain  of  Events 


1.  The  contractor  has  met  (or  exceeded)  an  MTBF  of  1000  hours. 

2.  There  is  (at  least)  a  0.90  probability  of  "passing"  the  test. 

3.  "Passing"  the  test  will  give  the  user  (at  least)  80%  confidence  that  the 
MAV  of  500  hours  MTBF  has  been  exceeded. 

4.  The  user  is  assured  that  his  needs  have  been  met. 

The  Negative  Chain  of  Events 

1.  The  contractor  has  met  an  MTBF  of  500  hours  (or  less). 

2.  There  is  (at  least)  a  0.80  probability  of  "failing"  the  test. 

3.  "Failing"  the  test  gives  the  procuring  activity  (at  least)  90%  confidence 
that  the  contractually  obligated  SV  of  1000  hours  MTBF  has  not  been  met. 


4.  The  procuring  activity  is  assured  that  the  contractual  obligations  have 
not  been  met. 


CASE  STUDY  NO.  6-2 


Background 

The  specified  MTBF  of  a  targeting  system  is  500  hours  and  the  minimum  accept¬ 
able  MTBF  is  400  hours.  The  contractor  has  proposed  a  development  test  con¬ 
sisting  of  6000  hours  on  the  initial  prototype  system  and  2000  hours  on  a 
second  prototype  system  which  will  contain  some  minor  engineering  advances. 

The  proposed  test  plan  of  8000  hours  can  distinguish  between  the  SV  of  600 
hours  and  the  MAV  of  400  hours  for  consumer's  and  producer's  risks  of  slightly 
over  20%.  If  the  producer  is  willing  to  accept  a  30%  producer's  risk,  the 
proposed  plan  will  yield  a  12%  consumer's  risk. 

Determine 

Comment  on  the  adequacy  of  the  proposed  test. 

t  *  “ 

Solution 


These  risks  seem  to  be  larger  than  should  be  considered  for  an  important 
system.  The  test  exposure  seems  to  be  inadequate  for  the  following  reasons: 

-  Test  time  is  not  of  sufficient  length. 

-  Prototypes  are  not  identical.  Test  time  on  the  second  prototype  may 
not  be  long  enough  to  determine  if  the  design  improvements  increase 
reliability. 

-  Only  two  systems  on  test  may  be  insufficient.  Ideally,  more  systems 
should  be  used  for  shorter  periods  of  time. 

A  test  plan  having  four  systems  accumulating  about  4000  hours  each  will  yield 
producer  and  consumer  risks  of  just  over  10%.  A  further  benefit  is  that  using 
four  systems  and  operating  them  for  a  period  of  time  about  10  times  the  mini¬ 
mum  MTBF  should  paint  a  pretty  clear  picture  of  the  system  capability  through¬ 
out  a  significant  part  of  its  expected  age. 

Note:  Chapter  8  will  present  the  analytical  tools  required  to  evaluate  the 
above  test  plan.  Our  objective  here  is  to  qualitatively  review  the 
various  aspects  of  a  statistically  relevant  test  program. 


CHAPTER  7 


RELIABILITY  DATA  ANALYSIS 


It  is  important  to  understand  that  for  any  realistic  situation  the  true  re¬ 
liability  characteristics  of  a  system,  or  fleet  of  systems,  are  never  known 
with  complete  certainty.  This  is  true,  of  course,  because  we  have  not,  in 
fact,  actually  tested  every  system  in  the  total  population  and,  practically 
speaking,  never  could.  To  compensate  for  this  lack  of  total  information,  some 
form  of  sampling  is  used  to  obtain  information  about  the  reliability  char¬ 
acteristics  inherent  in  a  system  and  to  quantify  the  level  of  uncertainty 
about  them.  Of  course,  uncertainty  continues  to  exist,  and,  as  a  consequence, 
the  reliability  parameters  can  only  be  estimated.  This  chapter  presents  pro¬ 
cedures  which  can  be  used  to  determine  estimates  for  the  various  reliability 
parameters  and  to  quantify  the  uncertainty  inherent  in  these  estimates. 

These  procedures  support  the  analysis  of  data  gathered  in  previously  conducted 
tests.  Planning  these  tests  to  assure  that  adequate  sample  sizes  are  obtained 
is  the  topic  of  Chapter  8.  The  objective  of  the  data  analysis  effort  is  to 
determine  "best  estimates"  of  system  performance  parameters,  such  as  reli¬ 
ability,  and  to  estimate  the  uncertainty  associated  with  these  "best  estimate" 
values . 

As  in  previous  chapters,  the  case  studies  illustrate  the  application  and 
manipulation  of  the  mathematical  concepts  presented  in  the  chapter  text.  Note 
that  in  the  typical  Chapter  7  case  study,  you  are  provided  the  results  of  a 
hypothetical  test  program  and  requested  to  develop  a  best  estimate  and  con¬ 
fidence  interval  for  a  reliability  parameter. 

Fixed  Configuration  and  Growth  Tests 

There  are  basically  two  types  of  reliability  tests.  One  is  a  test  of  fixed 
configuration.  The  other  is  the  growth,  or  developmental,  test,  which  centers 
on  reliability  improvement  seen  as  the  configuration  changes  during  the  test. 
There  is  not,  however,  a  clean  line  between  these  two  types.  For  the  truly 
fixed  configuration  test  of  continuously  operated  systems,  any  changes  in 
reliability  are  due  to  the  inherent  characteristics  of  the  hardware  and  how  it 
is  maintained.  The  analysis  is  done  as  a  function  of  system  age.  If  there 
are  design  changes,  they  have  to  be  considered  on  a  separate  basis,  perhaps  by 
a  data  set  for  each  configuration.  See  Chapter  10  for  more  details  on  this 
procedure. 

For  the  growth  type  of  test,  the  statistical  models  currently  available  assume 
that  all  changes  in  reliability  are  attributable  to  the  design  changes.  In 
other  words,  they  assume  that  the  inherent  reliability  of  the  hardware  is 
constant.  The  basic  analysis  for  the  growth  type  of  test  is  done  as  a  func¬ 
tion  of  test  exposure,  rather  than  age,  since  it  is  test  exposure  that  pro¬ 
vides  information  for  design  improvements.  The  effects  of  system  age  can  be 
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dealt  with  separately,  primarily  by  considering  the  failure  modes  that  are 
observed.  Chapte*-  9  summarizes  the  topic  of  reliability  growth  and  il¬ 
lustrates  the  associated  analysis  techniques. 

Discrete  and  Continuous  Tests 


The  most  elementary  consideration  in  beginning  a  data  analysis  is  to  determine 
whether  test  time  is  measured  continuously  or  discretely.  Usually,  this 
distinction  is  quite  obvious.  An  example  of  a  test  which  can  be  analyzed 
either  way  is  the  following.  Suppose  that  a  system  has  a  durability  require¬ 
ment  of  5000  hours  and  ten  systems  are  available  for  testing.  Each  system  is 
tested  until  it  either  experiences  a  durability  failure  or  successfully  com¬ 
pletes  the  5000  hour  test  period.  We  can  let  each  system  be  a  test  unit  and 
count  as  a  failure  any  system  which  fails  before  5000  hours.  This  is  a  dis¬ 
crete  time  approach.  Alternatively,  we  could  let  hours  be  our  test  units, 
with  the  total  operating  hours  of  the  10  systems  as  the  test  exposure.  This 
is  a  continuous  time  approach.  Another  example  is  the  firing  of  an  automatic 
weapon,  where  many  rounds  are  fired.  This  is  a  one-shot,  discrete  time  test 
if  we  are  analyzing  the  ammunition,  but  could  be  considered  a  continuous  time 
test  if  we  are  analyzing  the  gun  or  any  of  its  components.  Generally,  when 
either  approach  is  appropriate,  more  information  is  obtained  from  the 
continuous  time  approach. 

DISCRETE  TIME  TESTING 

Suppose  that  the  systems  under  test  are  single-shot  systems.  Each  test  unit 
results  in  a  distinguishable  success  or  failure.  As  discussed  in  Chapter  5, 
the  binomial  model  will  be  used  to  represent  or  model  system  reliability  when 
discrete  time  or  success/fail  operations  are  of  interest.  It  is  assumed 
throughout  this  discussion  on  discrete  time  testing  that  the  conditions  of  a 
binomial  model  are  reasonably  satisfied.  (See  Chapter  5.)  We  present  data 
analysis  for  success/fail  (discrete)  tests  in  the  form  of  point  estimates, 
confidence  intervals,  and  tests  of  hypotheses. 

Binomial  Model:  Point  Estimate  of  Failure  Probability 

Once  the  number  of  trials  has  been  specified  (see  Chapter  8),  all  the  informa¬ 
tion  contained  in  a  binomial  experiment  rests  in  the  number  of  failures  that 
occur.  We  use  this  information  to  make  an  assessment  or  an  estimate  of  the 
true  probability  of  failure,  p.  Thus,  our  best  estimate  of  the  value  of  p  is 
the  ratio  of  the  number  of  failures  to  the  number  of  trials.  This  ratio  is 
called  the  cample  proportion  of  failures  and  is  designated  by  the  syabol  p, 
called  p-hat.  We  use  this  sample  proportion  of  failures,  p,  to  construct 
confidence  intervals  for  p  and  in  testing  hypotheses  about  p.  By  definition, 
then 


a  number  of  failures  ,  . .  x  r  ., 

p  =  - r - —  =  sample  proportion  of  failures 

r  number  of  trials  r  r  r 

p  =  best  estimate  for  p 

p  ~  true  proportion  of  failures 


Note  that  true  system  reliability  is  the  probability  of  successful  operation, 
therefore 

R  =  1  -  p,  where  R  is  true  system  reliability,  and 

a  A 

R  =  1  -  p  =  best  estimate  of  system  reliability. 

It  is  important  that  the  test  designer  and/or  evaluator  understand  that  a 
point  estimate  for  p  represents  a  small  portion  of  the  information  contained 
in  the  data  generated  by  a  binomial  experiment.  Other  useful  information 
includes  upper  and  lower  confidence  limits  for  the  unknown  parameter,  p. 

Binomial  Model:  Confidence  Limits  for  Failure  Probability 

Confidence  limits  and  thei r  interpretation  should  play  &  vital  role  in  de¬ 
signing  and  evaluating  a  binomial  experiment.  Not  only  does  the  actual  in¬ 
terval  relay  a  significant  amount  of  information  about  the  data,  but  also  the 
method  of  interval  construction  can  aid  the  test  designer  in  determining 
adequate  test  exposure  to  meet  his  needs.  An  extensive  discussion  on  tbe 
interpretation  of  confidence  intervals  is  given  in  Chapter  6. 

Suppose  that  we  observe  "s"  failures  out  of  “a"  trials  in  a  binomial  experi¬ 
ment.  This  translates  to  a  sample  proportion  of  failures  equal  to  s/n  and  a 
sample  proportion  of  successes  equal  to  (n-s)/n.  Tables  of  exact  confidence 
limits  for  the  true  proportion  of  failures  for  values  of  n  less  than  or  equal 
to  30  are  given  in  Appendix  B,  Table  4.  As  an  example,  suppose  that  n  =  25 
trials  and  s  -  4  failures.  A  90%  upper  confidence  limit  for  p  is  0.294.  We 
obtaiu  this  value  using  Appendix  B,  Table  4  with  n  =  25  in  the  column  labeled 
90%  upper  limit  and  the  row  labeled  s  ~  4.  For  the  same  d.-  ta,  a  98%  con¬ 
fidence  interval  is 


0.034  <  p  <  0.398. 

In  this  case,  the  values  are  found  in  the  columns  labeled  98%  interval  and  the 
row  labeled  s  =  4.  More  examples  using  Table  4  are  given  in  Case  Study  7-3. 

Binomial  Model:  Confidence  levels  for  Pre-Established  Reliability  Limits 


If,  after  conducting  a  test  in  which  we  observed  s  failures  (c  =  n-s  suc¬ 
cesses)  out  of  n  trials,  we  wish  to  determine  how  confident  we  are  that  a 
pre-established  level  of  reliability  (such  as  the  MAV)  has  been  met  or  ex¬ 
ceeded,  we  may  use  equation  ?.l  below. 


Let  R^  designate  the  desired  pre-established  level  of  reliability.  To  find 
the  confidence  that  R.  has  been  met  or  exceeded,  we  evaluate  the  expression: 


(7.1) 


If  we  denote  the  value  of  this  expression  as  1  -  a,  then  we  are  100(1  -  Cf)% 
confident  that  R  >  R_ . 
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If,  on  the  other  hand,  we  wish  to  determine  how  confident  we  are  that  a  pre- 
established  level  of  reliability  (such  as  the  SV)  has  not  been  attained,  we 
stay  use  equation  7.2. 

Let  K,,  designate  the  desired  pre-established  level  of  reliability.  To  find 
the  confidence  that  !v.  has  not  been  attained,  we  evaluate  the  expression: 

WC)  =  jP  Ruk(1'h)n'k  (7'2) 

If  we  denote  the  value  of  this  expression  as  a,  then  we  are  100(1  -  a)%  con¬ 
fident  that  R  <  Ry. 

See  Case  Study  7-1  for  an  example  of  this  technique. 


The  Greek  letter  a  is  used  numerous  times 
throughout  this  chapter  to  represent  a  general¬ 
ized  value  or  designation  of  "RISK."  In  this 
chapter,  a  is  not  necessarily  to  be  interpreted 
as  producer's  risk  as  in  Chapters  6  and  8. 


Approximate  Binomial  Confidence  Limits  (Normal  Approximation) 

If  the  number  of  failures  and  the  number  of  successes  both  are  greater  than  or 
equal  to  5,  we  can  obtain  approximate  confidence  limits  using  the  normal 
distribution.  The  approximate  100(1 -a)%  lower  limit  for  p,  the  true  propor¬ 
tion  of  failures,  is 

P  >  PL 

P  >  P  -  zaVp(l“p)/n  ,  (7.3) 

where  p  =  s/n.  The  approximate  100(l-a)%  upper  confidence  limit  for  p  is 

P  iPu 

p<  p+  zaVp(l*p)/n  •  (7. 4) 

The  two-sided  100(l-c0%  confidence  limits  for  p  are 

PL  <  P  <  Pu 

P  -  za/2vP(1"P^/n  ~  p  -  P  +  za/2^P^”P)/n  •  (7-5) 
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Values  for  z and  Zg^  are  obtained  from  Appendix  B,  Table  2. 

As  an  example,  suppose  that  5  failures  in  30  trials  occurred  during  a  test. 
An  approximate  90%  (a  -  0.10)  upper  limit  for  the  true  proportion  of  failures 
is 


P  +  zo.io VpTl-pI/n  • 

Substituting  n  =  30,  and  p  =  5/30  =  0.166,  we  obtain 
0.166  +  Zq  10V(0."l66)  (0.834)/30  . 

The  value  of  zQ  is  determined  using  Appendix  B,  Table  2.  Under  the  column 

labeled  P(Z  >  z^)  we  search  the  values  until  we  find  the  number  closest  to 
0.10,  the  value  of  a.  The  number  in  the  column  labeled  zff  is  the  desired 
value.  In  this  case,  for  a  =  0.10,  z^  =  1.28.  The  upper  limit  is  then 

0,166  +  1.28^07l§6T^O34T730  , 

which  reduces  to  0.253.  We  are  thus  90%  confident  that  uhe  true  proportion  of 
failures  is  0.253  or  smaller. 

See  Case  Study  7-3  for  construction  of  confidence  limits  using  normal  approxi¬ 
mation. 

Approximate  Binomial  Confidence  Limits  (Poisson/Exponential  Approximation) 

When  the  sample  proportion  of  failures  is  small,  and  the  number  of  trials  is 
reasonably  large— at  least  30— we  can  obtain  approximate  confidence  limits 
using  techniques  described  in  the  section  on  Exponential  Model:  Confidence 
Intervals  and  Limits  for  MTBF.  This  is  an  especially  useful  technique  for 
situations  involving  very  few  failures  in  fairly  large  samples.  We  use  the 
procedure  for  failure  terminated  testing  with  the  identifications:  T  =  n  (the 
number  of  trials)  and  r  =  s  (the  number  of  failures).  We  obtain  approximate 
confidence  limits  for  p,  the  probability  of  failure,  by  constructing  confi¬ 
dence  limits  for  6,  the  system  MTBF.  Because  p  and  A  are  failure-oriented 
parameters  and  6  is  a  success-oriented  parameter  (remember  that  by  definition 
0  =  1/A) ,  an  approximate  confidence  limit  for  p  is  the  reciprocal  of  the 
confidence  lim?'  for  6.  An  important  consequence  of  the  reciprocity  mentioned 
above  is  that  upper  confidence  limit  for  6  yields  a  lower  confidence  limit 
for  p  and  vice  versa. 

Consider  the  situation  described  in  Chapter  6,  where  3  failures  out  of  30 
trials  of  a  binomial  experiment  were  observed.  To  construct  an  approximate 
90%  confidence  interval  for  the  true  proportion  of  failures,  we  let  T  be  30 
and  r  be  3.  The  95%  confidence  interval  for  0  is 

2T...  <  e  <  2T - 

*a/2,2r  *l-a/2,2r 
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since  T  =  n  =  30,  r  =  s  =  3,  and  a  =  0.05,  we  have 


■  <  e  < 

X0.025,6  X0.975,6 

2  2 

Values  for  Xq  Q25  g  and  Xq  g  are  obtained  fron  Appendix  B,  Table  5.  The 

explanation  of  how  to  extract  these  values  is  presented  below  in  the  section 
entitled  "Exponential  Model:  Confidence  Intervals  and  Limits  for  MTBF."  The 
2  2 

values  are  Xq  Q25  g  =  14*46  and  Xq  g  -  1.24.  Thus  the  interval  for  0  is 


msi  <  e  <  2122 1 

14.46  -  -  1.24  ’ 


which,  upon  simplification,  becomes 
4.15  <  0  <  48.39  . 

Taking  the  reciprocals  of  the  limits  for  0,  we  have  that  the  approximate  95% 
confidence  interval  for  the  true  proportion  of  failures  is 

0.021  <  p  <  0.241  . 

Since  reliability  is  1  -  p,  the  approximate  95%  confidence  interval  for  system 
reliability  is 

0.759  <  R  <  0.979  . 

This  statement  can  also  be  interpreted  as  follows:  We  are  95%  confident  that 
the  true  system  reliability  is*  between  0.759  and  0.979.  This  interval  is 
based  on  our  test  results  where  3  out  of  30  trials  ended  in  failure. 

See  Case  Study  7-2  for  another  example  of  this  procedure. 

Point  Estimates  and  Confidence  Limits  for  the  Difference/Ratio  of  Proportions 

Suppose  that  tests  have  been  conducted  on  two  different  types  of  systems  re¬ 
sulting  in  sample  proportions  of  failures  of  p^  and  Pj  with  sample  sizes  of  n^ 

and  n^,  respectively.  The  point  estimates  for  the  difference  (p^  -  p„)  and 
ratio  (p^/P2)  of  proportions  are  the  difference  and  ratio  of  the  sample  pro¬ 
portions,  i.e.,  Pj  -  p^  and  respectively.  We  present  the  procedures 

for  determining  confidence  limits  for  the  difference  and  for  the  ratio  of  the 
two  population  proportions  (p^  and  P2)  using  the  normal  distribution.  The  ap¬ 
proximate  100(1  -  o)%  lower  confidence  limit  for  the  true  difference  in  pro¬ 
portions  is 

P1  “  P2  -  (pi  "  ^L 


.  ^  A 

>  Pi  -  P2 


V'“l 


-  i-  .i :  Jk «\rl 
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The  approximate  100(1  -  a)%  upper  confidence  limit  for  the  true  difference  in 
proportions  is 


P1  "  P2  -  (P1  ’  P25U 

-  P1  ~  p2  +  Za^pl^l’pl)/al  +  P^-P^/*^  • 

The  approximate  100(1  -  a)%  confidence  interval  for  the  true  difference  in 
proportions  is 

(Pi  "  P2)L  <  PX  “  P2  <  (Pj  -  P2)„ 


P1  "  p2  “  Za/2Vpl(1"Pl)/al  +  p2(1_p2)/n2  1  pl  *  p2 

-  P1  “  p2  +  za/2Vpl(1“pl)/al  +  p2(1’p2)/n2  * 

With  high  reliability  systems,  it  is  sometimes  more  informative  for  comparing 
two  systems  to  look  at  the  ratio  of  proportions  of  failures.  As  an  example, 
suppose  that  the  true  proportions  of  failures  for  two  systems  are  0.01  and 
0.001.  We  can  say  that  one  system  is  ten  times  better  than  the  other  even 
though  the  difference  is  a  mere  0.009.  An  approximate  100(1  -  a)%  lower 
confidence  limit  for  the  true  ratio  of  proportions  is 

Pj/Pj  >  (Pj/P^l 


Pj/?2  £  Pj/p2  -  z0Vp1(1-P2^i12P2  • 

The  approximate  100(1  -  a)%  upper  confidence  limit  for  the  true  ratio  of 
proportions  is 

Pj/P2  <  (Pi/P2)u 

Pl/P2  <  Vp2  +  za^pl(1_p2)/n2p2  ‘ 

The  approximate  100(1  -  ct)%  confidence  interval  for  the  true  ratio  of  propor¬ 
tions  is 


Cpi/p2)L  <  Pj/P2  1  (Pi/P2)u 

pl/p2  *  Z0/2^Pl(1"P2)/n2P2  -  Pl/P2 
i  pl'p2  +  za/2^pl(1-p2)/n2^2  * 

In  Case  Study  7-4,  we  construct  confidence  limits  for  the  difference  and  ratio 
of  population  proportions. 
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Suppose  the  systems  under  test  operate  as  a  function  of  hours,  kilometers,  or 
other  continuous  measure.  In  such  a  case,  the  data  are  not  solely  success/ 
failure  oriented.  Generally,  the  times  at  which  failures  occur  and  the  time 
in  operation  without  failures  must  also  be  considered.  These  types  of  tests 
are  analyzed  by  using  a  Poisson  model.  When  the  failure  rate  remains  constant 
throughout  the  test,  the  exponential  distribution  describes  the  times  between 
failures  and  provides  all  the  information  needed  for  the  data  analysis.  For 
the  analysis  presented  in  subsequent  sections  of  this  chapter,  we  will  assume 
that  the  failure  rate  is  constant.  We  present  below  a  graphical  procedure  to 
determine  if  that  assumption  is  reasonable. 

Continuous  Time  Testing:  Failure  Pattern  Identification 

When  confronted  with  data  from  a  continuous  time  test  the  analyzer  should 
first  construct  an  average  failure  rate  plot.  The  purpose  of  constructing  an 
average  failure  rate  plot  is  to  help  the  analyst  determine  whether  the  failure 
rate  is  increasing,  decreasing,  or  is  constant.  The  type  of  failure  rate  plot 
that  will  be  described  considers  hardware  that  did  not  have  significant  design 
changes  made,  so  that  changes  in  the  failure  rate  are  due  primarily  to  the  age 
of  the  equipment.  (When  substantial  design  changes  are  made,  there  may  be 
reliability  growth.  In  that  case,  a  different  type  of  average  failure  rate 
plot  is  used,  which  is  based  on  cumulative  test  exposure  rather  than  the  age 
of  the  equipment.) 

The  average  failure  rate  plot  is  constructed  as  follows: 

1.  Determine  the  lowest  and  highest  equipment  ages  which  the  test  experience 
covers.  These  need  not  be  ages  at  which  failures  occurred.  This  estab¬ 
lishes  the  lower  and  upper  limits  of  the  plot.  For  convenience,  working 
limits  may  be  set  at  "round"  numbers  above  and  below  the  lower  and  upper 
limits,  respectively. 

2.  Divide  the  interval  encompassed  by  the  working  limits  into  subintervals. 
The  subintervals  need  not  be  of  equal  size. 

3.  Count  the  number  of  failures  in  each  subinterval.  (A  minimum  of  5  fail¬ 
ures  per  subinterval  is  desirable,  though  not  absolutely  necessary.) 

4.  Add  up  the  hours  (or  miles,  rounds,  etc.)  of  operation  within  each  sub¬ 
interval  . 

5.  Compute  the  average  failure  rate  for  each  subinterval  by  dividing  the 
number  of  failures  in  the  subintervsl  by  the  hours  (or  miles,  rounds, 
etc.)  of  operation  in  the  subinterval. 

6.  Construct  a  jraph,  with  the  system  age  (in  hours,  miles,  rounds,  etc.)  on 
the  horizontal  scale,  and  failure  rate  on  the  vertical  scale.  The  aver¬ 
age  failure  rates  computed  for  each  subinterval  are  shown  as  horizontal 
lines  over  the  length  of  each  subinterval. 

7.  If  the  average  failure  rate  plot  has  too  much  fluctuation  to  show  any 
kind  of  trend,  reduce  the  number  of  subintervals  and  repeat  steps  3 
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through  6.  For  very  small  amounts  of  data,  it  may  be  necessary  to  use 
only  two  subintervals. 

8.  From  the  final  version  of  the  average  failure  rate  plot,  judge  whether 
the  failure  rate  trend  remains  constant,  increases,  or  decreases  as  the 
equipment  ages.  For  small  amounts  of  data  it  may  be  difficult  to  make 
this  judgment.  In  any  case,  statistical  tests  for  trend  may  be  used. 

9.  If  the  data  are  judged  to  have  no  trend,  analyses  based  on  the  exponent 
,  tial  distribution  may  generally  be  used  with  validity. 

10.  If  the  failure  rate  is  judged  to  be  increasing  or  decreasing,  as  a  mini¬ 
mum,  a  note  to  this  effect  should  accompany  any  analyses  based  on  the  as¬ 
sumption  of  exponential  times  between  failures.  To  analyze  data  that 
appear  to  have  a  trend  more  explicitly,.,  a  non-homogeneous  Poisson  process 
may  be  fitted  to  the  data.  We  do  not  present  any  analysis  using  a  non- 
homogeneous  Poisson  process  in  this  chapter.  If  the  average  failure  rate 
plot  indicates  that  a  constant  failure  rate  assumption  is  unwarranted, 
the  data  analyst  may  refer  to  a  statistics  text  which  covers  the  topic  of 
stochastic  processes  in  depth  to  aid  in  his  analysis. 

11.  See  Case  Studies  7-5  and  7-6  for  examples  of  average  failure  rate  plots. 

Exponential  Model;  Point  Estimate  of  MTBF 

When  data  are  judged  to  show  a  constant  failure  rate,  the  exponential  distri¬ 
bution  may  be  used  for  data  analysis.  Exponential  analysis  does  not  require 

the  use  of  actual  failure  times. 


Notation 


Formulas 


T  =  total  test  exposure,  the  total  hours,  miles, 

etc.,  accumulated  among  all  the  items  included 
in  the  sample 

r  =  number  of  failures  observed 

6  =  point  estimate  of  MTBF 

&(x)  =  point  estimate  of  reliability  for  a  specified 

exposure,  x 

X  =  the  point  estimate  of  the  failure  rate 

A  t 

6  =  r  (7.6) 


Exponential  Model:  Point  Estimates  of  Reliability  and  Failure  Rate 


Point  estimates  of  reliability  and  failure  rate  may  be  developed  from  point 
estimates  of  MTBF  as  follows: 


R(x)  »  «',/e 


(7.7) 
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X  =  i/e  (7.8) 

See  Case  Studies  7-7,  7-8,  and  7-9  for  illustrations  of  computing  point  esti¬ 
mates. 


Exponential  Model:  Confidence  Intervals  and  Limits  for  MTBF 


Notation  T 


r 


total  test  exposure,  the  total  hours,  miles, 
etc.,  accumulated  among  all  the  items  included 
in  the  sample 

the  number  of  failures  observed 


2 

X _ , _  =  a  chi-square  value,  identified  by  two  sub¬ 

scripts.  To  determine  a  chi-square  value  using 
Appendix  B,  Table  5,  we  use  the  first  subscript, 
a  function  of  the  risk  (a) ,  to  indicate  the 
column,  and  the  second  subscript,  a  function  of 
the  number  of  failures  (r),  to  indicate  the  row. 


a  =  the  risk  that  a  confidence  statement  is  in 
error.  Note:  The  symbol  a  used  here  does  not 
necessarily  represent  the  producer's  risk  as 
discussed  in  Chapter  6. 


0  =  MTBF 


R(x)  =  reliability  for 
a  period  x 


.  no  subscript  =  true 
I  but  unknown  value 
Used  in  | 

conjunction  <  L  subscript  =  lower  limit 
with  I 


K  =  failure  rate 


U  subscript  -  upper  limit 
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Formulas  (All  the  formulas  listed  below  will  yield  statements  at  the 
100(l-a%)  level  of  confidence.) 


Confidence  Limits 


Time  Terminated 


Failure  Terminated 


When  the  test  exposure  ends  at  a  time 
other  than  a  failure  occurrence,  use 
Appendix  B,  Table  8a  multipliers  or 
the  following  formulas. 


Interval  for  specified  confidence 
level 

0L  -  0  -  0U 

-y-— -  <  6  <  —21 -  (7.9a) 

*a/2,2r+2  *l-ot/2,2r 

See  Case  Studies  7-7  and  7-8. 

Lower  limit  for  specified  confi¬ 
dence  level 

e  >eL 

0>_2I_  (7.10a) 

*a,2r+2 


Upper  limit  for  specified  confi¬ 
dence  level 

0±0u 

0  <  2-2-  (7.11a) 

Xl-0t,2r 

See  Case  Study  7-7. 


When  the  test  exposure  ends  at  a 
failure  occurrence,  use  Appendix 
B,  Table  8b  multipliers  or  the 
following  formulas. 


Interval  for  specified  confidence 
level 

0L<0<eU 

2T  .  „  .  2T 

2  -  -  2 

Xa/2,2r  Xl-ot/2,2r 

(7.9b) 

See  Case  Study  7-9. 

Lower  limit  for  specified  confi¬ 
dence  level 

0  >  0L 

e>  ” 

*a,2r 

(7.10b) 

Upper  limit  for  specified  confi¬ 
dence  level 


0  <  8U 

0  <  _JL_  (7.11b) 

*l-a,2r 

See  Case  Study  7-9. 


( 
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Confidence  That  A  Specific  Lower  Limit  Has  Been  Attained 


Time  Terminated 


Confidence  that  a  specific  lower 
limit,  8r ,  has  been  attained 


,2  -21 
\*,2r+2  0L 


(7.12a) 


Search  tables  in  row  labeled 
2r  +  2  for  the  numerical  value, 
2T/0.,  and  find  the  associated 
value  for  a. 

Confidence  that 


e  >  eL 


is  100(l-«)%. 

The  value,  cr,  may  also  be 
determined  in  closed  form 
as  follows: 


Failure  Terminated 


Confidence  that  a  specific  lower 
limit,  6^,  has  been  attained 


2  _  2T 

'0f,2r  0. 


(7.12b) 


Search  x  tables  in  row  labeled 
2r  for  the  numerical  value, 
2T/6^,  and  find  the  associated 
value  for  or. 

Confidence  that 


e  >  eL 


is  10O(l-ct)%. 

The  value,  a,  may  also  be 
determined  in  closed  form  as 
follows : 


r  (T/0.)k  e“(T/V 

a  =  2  - - — - -  (7.13a) 

k=0 

(Use  Appendix  B,  Table  3  or 
Chart  1  to  evaluate  this  ex¬ 
pression.  ) 

Confidence  that 


0  >  eL 


r-l  (T/6.)k  e"(T/V 

a  =  l  - — jri -  (7.13b) 

k=0 

(Use  Appendix  B,  Table  3  or 
Chart  1  to  evaluate  this  ex¬ 
pression.) 

Confidence  that 

0  >  0T 


is  100(l-ct)%. 

See  Case  Studies  7-7  and  7-8. 


is  I00(l-a)%. 

See  Case  Study  7-9, 


Confidence  That  A  Specific  Upper  Limit  Has  Not  Been  Attained 


Time  Terminated 


Failure  Terminated 


Confidence  that  a  specified  upper 
limit,  Qy,  has  not  been  attained 


v2  =  2T 

*l-a,2r  By 


(7.14a) 


Confidence  that  a  specified 
upper  limit,  0^,  has  not  been 
attained 


,2  2T 

Cl-0f,2r  "  6ti 


(7.14b) 


Search  x"  tables  in  the  row  labeled 
2r  for  the  numerical  value,  2T/0.  , 
and  find  the  associated  value  for  1-ct. 


Search  x  tables  in  the  row 
labeled  2r  for  the  numerical 
value,  2T/0jj,  and  find  the 
associated  value  for  1-a. 


Confidence  that 


Confidence  that 


0  <eu 


0  5  eu 


is  100(l-a)%. 


is  lOO(l-ot)%. 


The  value  a  may  also  be  determined 
in  closed  form  using  the  following 
equation: 


The  value  a  may  also  be  deter¬ 
mined  in  closed  form  using  the 
following  equation: 


r=l  (T/0,  )k  e”(T/V 
i-«-  *  — - 

k=0 


(7.15a) 


r-1  (T/0n)k  e"(T/V 

1-a  =  Z  - j-i -  (7.15b) 

k=0  1 


(Use  Appendix  B,  Table  3  or 
Chart  1  to  evaluate  this  ex¬ 
pression.  ) 


(Use  Appendix  B,  Table  3  or 
Chart  1  to  evaluate  this  ex¬ 
pression.) 


Confidence  that 


Confidence  that 


6  <  0U 


0  <  9u 


is  100(l-a)%. 

See  Case  Study  7-7. 


is  100(l-a)%. 

See  Case  Study  7-9. 


Exponential  Model:  Confidence  Intervals  and  Limits  for  Reliability  and  Fail¬ 
ure  Rate 

Intervals  for  reliability  and  failure  rate  with  100(l-Cf)%  confidence  are 
Rl(x)  <  R(x)  <  R^x) 


-(x/eL) 


<  R(x)  <  e 


-Cx/Sy) 


and 


*1  -  X  -  \j 

1/0JJ  <  x  <  i/eL 


(7.16) 


(7.17) 


where  0^  and  0^  are  the  lower  and  upper  limits  of  the  100(l-a)%  confidence 
interval  for  0  (MTBF) . 

Lower  limit  for  reliability  and  upper  limit  for  failure  rate  with  100(l-c<)% 
confidence  are 


R(x)  >  RjU) 


-(x/0t) 
R(x)  >  e  L 


and 


(7.18) 


X  -  \j 

X  <  1/0L  (7.19) 

where  0^  is  the  100(l-Cf)%  lower  confidence  limit  for  0  (MTBF). 

Upper  limit  for  reliability  and  lower  limit  for  failure  rate  with  100(l-a)% 
confidence  are 


R(x)  <  Rjj(x) 


R(x)  <  e 


-Cx/Sy) 


(7.20) 


X>  Xl 
X  >  l/Ojj 

Where  ft,  is  the  100(l-ot)%  upper  confidence  limit  for  0  (MTBF). 


(7.21) 


1 


A  1 1  £&  \  Vv" 
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CASE  STUDY  NO.  7-1  - 


Background 

The  engine  for  a  light  armored  vehicle  must  have  a  0.90  probability  of  com¬ 
pleting  100,000  miles  without  an  operational  durability  failure.  In  order  to 
evaluate  durability,  four  vehicles  are  tested.  Each  vehicle  is  operated  until 
a  durability  failure  occurs  or  until  it  successfully  completes  100,000  miles 
of  operation  without  experiencing  an  operational  durability  failure. 

Determine 

1.  If  no  failures  occur,  what  confidence  do  we  have  that  the  requirement  has 
been  met  or  exceeded? 

2.  If  1  failure  occurs,  what  confidence  do  we  have  that  the  probability  is 
at  least  0.75? 

3.  If  2  failures  occur,  what  confidence  do  we  have  that  the  probability  is 
at  least  0.50? 

Solution 


1.  Since  no  failures  have  occurred,  the  number  of  successes  is  4.  We  use 
equation  7.1  with 

n  =  4  * 

s  =  4 
=  0.90. 


The  confidence  is: 

1  (£)(0.9)k(0.1)4“k  =  (o)(0’9)°(0. I)4  +  (i)c°*9)1(0. I)3 
k-0 

+  (i^0-9)^0-1)2  +  (^(o.^co.i)1 

=  (1)(0.0001)  +  (4) (0.0009)  +  (6) (0.0081)  +  (4)(0.0729) 
=  0.0001  +  0.0036  +  0.0486  +  0.2916  =  0.3439. 

We  are  34%  confident  that  the  reliability  meets  or  exceeds  0.90. 
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2.  The  number  of  successes  is  3.  We  use  equation  7.1  with 

n  =  4 
a  =  3 
=  0.75. 

The  confidence  is: 

1  (J)(0.75)k(0.25)4“k  -  0.2617. 

We  are  26%  confident  that  the  reliability  meets  or  exceeds  0.75. 

3.  The  number  of  successes  is  2.  We  use  equation  7.1  with 

n  -  4 
s  =  2 
=  0.5. 

The  confidence  is: 

X  (t)(0.5)k(0.5)4~k  =  0.3125.  * 

fc=0' 

We  are  31%  confident  that  the  reliability  meets  or  exceeds  0.50. 

Commentary 

It  is  interesting  to  note  that  with  the  small  sample  size,  we  can  only  reach 
34%  confidence  that  the  requirement  has  been  met  or  exceeded,  even  though  we 
encountered  zero  failures.  In  many  cases,  durability  requirements  are  im- 
possible  to  demonstrate  at  high  confidence  levels  because  sample  sizes  are 
almost  always  constrained  to  be  small. 
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CASE  STUDY  NO.  7-2 


Background 

A  launcher  for  a  medium  range  anti-tank  missile  has  been  tested.  Of  100 
missiles^  95  were  launched  successfully. 

Determine 

1.  Point  estimate  of  reliability. 

2.  Construct  a  90%  upper  limit  on  the  true  proportion  of  failures  using  the 
Poisson/exponential  approximation. 

3.  Construct  an  80%  confidence  interval  on  the  true  reliability  using  the 
Poisson/ exponential  approximation. 

Solution 


1.  Point  estimate  of  p,  the  true  proportion  of  failures  is  5/100  =  0.05. 
Consequently,  the  point  estimate  for  the  reliability,  R,  is 

6  =  1  -  p  =  1  -  0.05  =  0.95. 

2.  We  set  T  =  n  =  100,  r  =  s  =5,  and  a  =  0.10.  The  approximate  90%  upper 
limit  for  p,  the  true  proportion  of  failures,  is  obtained  by  first  determining 
a  90%  lower  limit  for  0.  The  90%  lower  limit  for  0  is 

0  >  0L 


Xa,2r 

>  2(100) 

-  15.99 

>  12.51. 

Consequently,  the  90%  upper  limit  for  p  is 

P  iPu 


<  — i — 

-  12.51 

<  0.08. 
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Thus,  we  are  90%  confident  that  the  true  proportion  of  failures  does  not 
exceed  0.08. 

3.  We  set  T  =  n  =  100,  r  =  s  =  5,  and  a  =  0.20.  The  80%  interval  for  R,  the 
launcher  reliability,  is  obtained  by  first  determining  an  80%  interval  for  0. 
The  80%  interval  for  6  is 

8L  <  6  <  «u 

2T  <  e  <  2T - 

2  —  —  2 

Xa/2,2r  Xl-a/2,2r 

2(100)  <  A  <  2(100) 

18.31  -  -  3.94 

10.92  <  6  <  50.76. 

Consequently,  an  80%  interval  for  p,  the  true  proportion  of  failures  is 

PL  -  P  -  PU 

1  .  .1 

*SSPi*E 

1  <  „  <  1 
50.76  -  p  -  10.92 

0.02  <  p  <  0.09. 

The  80%  interval  for  the  reliability,  R,  is 

rl  1  R  <  Ru 

1  -  Pu  <  R  <  1  -  PL 

1  -  0.09  <  R  <  1  -  0.01 
0.91  <  R  <  0.98. 

We  are  80%  confident  that  the  true  launcher  reliability  is  between  0.91  and 
0.98. 
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CASE  STUDY  NO.  7-3 


Background 

A  new  missile  system  has  been  under  development  and  is  ready  for  production. 
The  contract  specifies  that  the  producer  must  demonstrate  a  proportion  of 
successes  at  least  equal  to  0.85  (SV).  The  user  will  accept  as  a  minimum  a 
demonstration  of  at  least  0.70  (MAV).  An  initial  production  test  of  30  fir¬ 
ings  was  conducted  for  the  missile  system,  and  6  missiles  fired  improperly. 

Determine 


1.  What  is  our  best  single  value  estimate  for  the  true  proportion  of  fail¬ 
ures? 


2.  Construct  exact  90%,  95%,  and  99.5%  lower  confidence  limits  for  the  true 
proportion  of  failures. 

3.  Construct  exact  90%,  95%,  and  99.5%  upper  confidence  limits  for  the  true 
proportion  of  failures. 

4.  Construct  approximate  60%,  70%,  80%,  and  90%  two-sided  confidence  limits 
for  the  true  proportion  of  failure j,  using  the  normal  approximation  to  the 
binomial . 

5.  Provide  an  accept/ reject  criterion  which  permits  the  greatest  number  of 
acceptable  failures  which  still  meets  a  consumer's  risk  of  no  more  than  10%. 
What  is  the  producer's  risk  for  this  criterion?  Is  the  system  acceptable 
under  this  criterion? 

6.  Increase  the  sample  size  to  40  and  50.  Provide  an  accept/reject 
criterion  to  meet  a  producer's  risk  of  15%.  What  is  the  consumer's  risk  for 
each  criterion? 

Solutions 


1.  Point  estimate:  6/30  =  0.20.  This  corresponds  to  an  80%  reliability. 

2.  Lover  confidence  limits:  Use  Appendix  B,  Table  4. 


L  . 

90% 

Lower 

limit,  n  =  30, 

s 

=  6. 

Lower 

limit  s  0.109. 

b. 

95% 

Lower 

limit,  n  =  30, 

8 

=  6. 

Lower 

limit  =  0.091. 

c. 

99.5% 

Lower 

limit,  n  =  30, 

s 

=  6. 

Lower 

limit  =  0.054. 

Note  that  the  three  solutions  above  are  lower  confidence  limits  on  the  true 
proportion  of  failures,  i.e.,  lower  limits  on  unreliability.  If  we  subtract 


any  of  the  lower  limits  from  1,  we  obtain  an  upper  limit  on  reliability.  To 
convert  the  90%  lower  limit  on  unreliability  (0.109)  to  an  upper  limit  on 
reliability,  we  subtract  it  from  1,  i.e.,  1  -  0.109  =  0.891.  This  means  that 
we  are  90%  confident  that  the  true  reliability  does  not  exceed  0.891. 

3.  Upper  confidence  limits:  Use  Appendix  B,  Table  4. 


a. 

90% 

Upper  limit,  n  =  30, 
Upper  limit  =  0.325. 

s  =  6. 

b. 

95% 

Upper  limit,  n  =  30, 
Upper  limit  =  0.357. 

s  =  6. 

c. 

99.5% 

Upper  limit,  n  =  30, 
Upper  limit  =  0.443. 

s  =  6. 

Note  that  the  three  solutions  above  are  upper  confidence  limits  on  the  true 
proportion  of  failures,  i.e.,  upper  limits  on  unreliability.  To  obtain  a 
lower  limit  on  reliability,  we  subtract  the  corresponding  upper  limit  on 
unreliability  from  1.  The  90%  lower  limit  on  reliability  is  thus:  1  - 
0.325  =  0.675.  This  means  that  we  are  90%  confident  that  the  true  reliability 
exceeds  0.675. 

4.  Approximate  two-sided  .limits  (normal),  for  p  =  s/n  =  6/30  =  0.2: 

Lower  limit  =  p  -  z^VpTT-pTTn 

Upper  limit  =  p  +  z^VpO-pJ/n 

Note  that  the  values  for  zay2  can  be  £ound  in  Appendix  B,  Table  To  use  the 

table  for  two-sided  limits,  we  convert  the  confidence  percentage  (say  60%)  to 
a  value  for  a(0.40),  divide  that  value  by  2(a/2  =  0.20),  and  locate  the  value 

for  za/2  (20.20  =  °-84)’ 

a.  60%  a  =  0.40  zu/2  =  zQ  2Q  =  0.84 

Lower  limit  =  0.139 
Upper  limit  =  0.261 

b.  70%  a  =  0.30  za/2  =  zoa5  *  I-0* 

Lower  limit  =  0.124 
Upper  limit  =  0.276 

c.  80%  a  =  0.20  zay2  =  zQ  1Q  =  1.28 


Lower  limit  =  0.107 
Upper  limit  =  0.293 


=  1.645 


90%  a  =  0.10 


za/2  =  z0.05 


6. 


Lower  limit  =  0.080 
Upper  limit  =  0.320 

a.  Use  Appendix  B,  Table  1,  n  =  30.  The  probability  of  5  or  fewer 
failures  when  p  is  0.3  is  0.0766.  (Recall  that  p  =  0.3  corresponds 
to  a  reliability  of  0.7.)  The  probability  of  6  or  fewer  failures 
when  p  is  0.3  is  0.1595.  Because  the  consumer's  risk  is  not  to 
exceed  10%,  we  must  make  our  decision  criterion  to  accept  with  5  or 
fewer  failures  and  reject  with  more  than  5  failures.  The  decision 
criterion  to  accept  with  6  or  fewer  failures  results  in  a  consumer's 
risk  of  15.95%,  which  exceeds  the  requirement  of  a  10%  consumer's 
risk.  Note  that  the  actual  consumer's  risk  for  the  criterion  to 
accept  with  5  or  fewer  failures  is  7.66%. 

b.  Use  Appendix  B,  Table  1,  n  =  30.  The  producer's  risk  is  the  prob¬ 
ability  of  rejecting  the  system  when  it  has  met  the  specification  of 
0.15  proportion  of  failures  (i.e.,  a  reliability  of  0.85).  We 
reject  the  system  if  6  or  more  failures  occur.  The  probability  of  6 
or  more  failures  is  the  difference  between  1  and  the  probability  of 
5  or  fewer  failures.  The  probability  of  5  or  fewer  failures  when  p 
is  0.15  is  0.7106.  Consequently,  the  producer's  risk  is  1  -  0.7106 
or  0.2894  (28.94%). 

c.  The  system  is  not  acceptable  because  in  fact  more  than  5  failures 
occurred. 

a.  Appendix  B,  Table  1,  n  =  40,  p  =  0.15.  Producer's  risk  must  not 
exceed  0.15. 


r  P(r  or  fewer  failures) 

7  0.7559 

8  0.8646 


P(r+1  or  more  failures) 
0.2441 
0.1354 


b. 


The  criterion  is  to  reject  if  9  or  more  failures  occur;  otherwise, 
accept. 

The  consumer's  risk,  the  probability  of  accepting  the  system  when, 
in  fact,  it  has  fallen  below  the  MAV  of  0.7,  is  the  probability  that 
8  or  fewer  failures  occur  when  the  true  proportion  of  failures,  p, 
is  0.3.  This  value  is  0.1110.  Thus,  there  is  an  11.1%  chance  of 
accepting  a  bad  system  with  this  plan. 

Appendix  B,  Table  1,  n  =  50,  p  =  0.15.  Producer's  risk  must  not 
exceed  0.15. 


r  P(r  or  fewer  failures) 
9  0.7911 

10  0.8801 


P(r+1  or  more  failures) 
0.2089 
0.1199 
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The  criterion  is  to  reject  if  11  or  more  failures  occur;  otherwise, 
accept. 

The  consumer's  risk  (note  above  definition)  is  the  probability  that 
10  or  fewer  failures  occur  when  p  is  0.3.  This  value  is  0.0789. 
Thus,  there  is  a  7.89%  chance  of  accepting  a  bad  system  with  this 
plan. 

Note  that  for  a  fixed  producer's  risk  (approximately  13%),  the 
consumer’s  risk  decreases  as  the  sample  size  increases.  An  in¬ 
creased  sample  size  will  also  result  in  a  decreased  producer's  risk 
when  the  consumer's  risk  is  held  approximately  constant. 
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CASE  STOPY  NO.  7-4 


Background 

Two  contractors  are  competing  for  a  contract  to  produce  an  electronic  guidance 
system.  Twenty-five  units  from  Contractor  1  and  thirty  units  from  Con¬ 
tractor  2  have  been  tested.  The  results  of  the  test  are:  Contractor  1  had  2 
failures,  Contractor  2  had  7  failures. 

Determine 


1.  What  is  the  point  estimate  of  the  difference  in  the  true  proportions  of 
failures  for  the  two  contractors? 

2.  What  is  the  point  estimate  of  the  ratio  of  the  true  proportions  of  fail¬ 
ures  for  the  two  contractors? 

3.  Construct  an  approximate  90%  lower  confidence  limit  for  the  difference  in 
the  true  proportions. 

4.  Construct  an  approximate  90%  lower  confidence  limit  for  the  ratio  of  the 
true  proportions. 

5.  What  confidence  do  we  have  that  Contractor  l's  system  is  at  least  twice 
as  bad  as  Contractor  2's?  At  least  50%  worse  than  Contractor  2's? 

Solutions 

1.  Pj  -  p2  =  7/30  -  2/25  =  0.233  -  0.080  -  0.153. 

2.  pj/p2  *  7/30  r  2/25  ■  (7)(25)/2(30)  =  2.91. 

3.  Lower  limit  =  P*  "  P2  “ 

=  0.153  -  zav(0.233)(0.767)/30  v  (0.08) (0.92)/25 

For  a  90%  lowe  limit,  a  =  0.10  and  =  1.28.  (See  Appendix  B,  Table  2.) 

The  lower  limit  far  the  difference  in  true  proportions  is  0.031.  This  means 
that  we  ure  90%  confident  that  the  difference  in  the  true  proportions  of  fail¬ 
ures  is  at  least  0  031. 

4.  Lower  limit  =  pj/p2  -  zaVplCl-p2)/n2P2 

=  2.91  -  *aV(0.233)(0.92)/25(0.08)2  . 
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For  a  90%  lower  limit,  a  =  0.10  and  ztf  =  1.28.  The  lower  limit  is  thus  1.43. 

This  means  that  we  are  90%  confident  that  Contractor  l's  system  is  1.43  times 
worse  than  Contractor  2's  system. 

5.  To  find  the  confidence  that  Contractor  l's  system  is  at  least  twice  as 
bad  as  Contractor  2's,  we  must  find  the  confidence  associated  with  a 
lower  limit  of  2  for  the  ratio.  Since,  by  definition,  the  lower  ltait  is 


Pl/P2  ‘  2a Vp i ( 1 "P2 ) / a2$2  * 

We  set  this  expression  equal  to  2,  and  solve  for  z^  to  obtain 


i0  •  <f,/J2  -  zWpjU-Pj)/"^  • 

Substituting  0.233  for  p^,  0.08  for  $2,  and  25  for  n2»  we  have 
za  =  0.788  . 

We  look  in  Appendix  B,  Table  2  to  find  the  value  of  a  which  corresponds  to  zfl 

*  0.788.  Since,  by  definition,  P(Z  >  zQ)  =  a,  the  desired  value  of  a  is 

located  under  the  column  labeled  P(Z  >  z) .  Thus  the  value  of  a  is  0.215. 

—  a 

This  represents  a  100(l-ot)%  =  78.5%  lower  confidence  limit,  so  we  are  78.5% 
confident  that  Contractor  l's  system  is  at  least  twice  as  bad  as  Con¬ 
tractor  2's. 

To  find  the  confidence  that  Contractor  l's  system  is  at  least  50%  worse  than 
Contractor  2's,  we  sobi*  the  following  equation  for  zQ, 

Vp2  '  za^ l(1“P2)/n2P2  =  1*5  ’ 

The  solution  is: 


2a  *  v  V*2  "  1<5)/Vp1(l-P2)/a2P2  * 


Substituting  0.233  for  Pj,  0.08  for  $2,  and  25  for  n2>  we  have 

z„  =  1.22  . 

0 
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tor  l's  systea  is  at  least  50%  worse  than  Contractor  2' a. 


i  'i 

a 


CASE  STUDY  NO.  7-5 


Background 

Six  electronic  systems  were  tested.  All  systems  had  the  same  configuration, 
and  no  design  changes  were  introduced  during  the  test.  The  test  experience  is 
tabulated  below. 


System  Age 

System  Age 

System  Age 

at  Start  of 

at  Failure(s) 

at  End  of 

System 

Test.  Hrs. 

Hrs. 

Test,  Hrs. 

1 

0 

13,  37,  60 

275 

2 

75 

154 

290 

3 

0 

73 

290 

A 

150 

190,  218 

270 

5 

0 

3,  52,  227 

260 

6 

0 

39,  166,  167,  209 

260 

Determine 

Is  exponential  data  analysis  appropriate  for  this  data  set? 
Solution 


An  average  failure  rate  plot  will  be  used  to  determine  if  there  is  a  trend  to 
the  data.  The  following  graph,  although  not  a  necessary  part  of  the  analysis, 
is  included  to  aid  visualization  of  the  data. 


SYSTEM  ill,  HOURS 


1 

4 


) 

1 


The  data  will  be  broken  down  into  three  equal  intervals.  The  steps  involved 


in  arriving  at  the 
following  table. 

average  failure  rate 

for  each  interval 

are  contained  in  the 

Average 

Interval 

Failures 

Operating  Hours 

Failure  Rate 

0-100 

7 

425 

7/425  =  0.0165 

100-200 

4 

550 

4/550  =  0.0073 

200-300 

3 

445 

3/445  =  0.0067 

These  average  failure  rates  are  plotted  on  the  following  graph. 


The  average  failure  rate  plot  suggests  very  strongly  that  there  is  a  decreas¬ 
ing  failure  rate  as  the  system  ages,  and  exponential  data  analysis  should  not 
be  used  unless,  at  a  minimum,  a  caveat  about  the  decreasing  failure  rate  is 
included . 

Commentary 

1.  Although  this  is  a  fictional  data  set,  the  pattern  to  the  data  is  fre¬ 
quently  observed  in  real  data  sets. 

2.  For  a  data  set  of  this  type,  it  is  generally  useful  to  consider  the 
actual  failure  types  and  corrective  actions  encountered.  This  tends  to 
clarify  how  permanent  the  high  initial  failure  rate  might  be. 


CASE  STUDY  NO.  7-6 


Three  developmental  vehicles  were  operated  under  test  conditions  that  closely 
aatched  the  operational  mode  summary  and  mission  profile  for  the  system.  All 
vehicles  had  the  same  physical  configuration.  Only  one  relatively  minor 
design  change  was  introduced  to  the  test  vehicles  during  the  test.  Scoring  of 
test  incidents  determined  that  there  were  7  operational  mission  failures.  The 
following  table  displays  the  operational  missies  failure  data. 


Vehicle 

Number 

1 

2 

3 


Odometer 


at  Start(km) 


0 

0 


0 


Odometer 


at  Failure (s) 


None 

3,721;  6,121;  6,175 
9,002 

216;  561;  2,804 


Odometer 


at  End 


6,147 

11,000 


5,012 


Determine 

Is  exponential  data  analysis  appropriate  for  this  data  set? 

Solution 

An  average  failure  rate  plot  will  be  used  to  determine  if  there  is  a  trend  to 
the  data.  Three  equal  intervals  will  (arbitrarily)  be  used. 


Interval 

Failures 

Kilometers 

Average  Failure  Rate 

0-4,000 

4 

12,000 

4/12,000  =  0.00033 

4,000-8,000 

2 

7,159 

2/7,159  =  0.00028 

8,000-12,000 

1 

3,000 

1/3,000  =  0.00033 
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Since  the  average  failure  rate  plot  is  essentially  horizontal,  there  is 
virtually  no  evidence  of  trend  in  tke  data,  and  exponential  data  aualysis 
procedures  may  be  used. 

Commentary 

For  large  data  sets,  the  average  failure  rate  plot  gives  a  very  precise 
picture  of  the  actual  failure  rate  pattern.  For  small  data  sets,  such  as  this 
one,  chance  plays  a  very  heavy  role.  For  example,  in  this  case  we  observed 
one  failure  in  the  last  interval.  Just  one  more,  or  one  less  failure  in  this 
interval  would  make  a  drastic  difference  in  the  observed  average  failure  rate. 
More  formal  trend  tests  address  whether  such  variations  could  reasonably  be 
due  to  chance  alone. 
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CASE  STUDY  NO.  7-7 


Background 

The  vehicle  system  discussed  in  Case  Study  7-\6  has  a  mission  duration  of  100 
kilometers.  The  user  has  stated  a  minimum  acceptable  value  (MAV)  of  2,00' 
mean  kilometers  between  operational  mission  failure  (MKBOMF)  and  the  con¬ 
tractual  reliability  requirement  is  equivalent  to  a  4,000  MKBOMF  specified 
value  (SV).  The  analysis  in  Case  Study  7-6  showed  no  trend  to  the  data.  The 
test  gave  22,159  km  of  exposure,  and  7  operational  mission  failures  were 
observed. 

Determine 


1.  Point  estimate  of  MKBOMF,  mission  reliability  and  failure  rate. 

2.  80%  confidence  interval  for  MKBOMF  and  mission  reliability. 

3.  80%  lower  confidence  limit  for  MKBOMF. 

4.  80%  upper  confidence  limit  for  MKBOMF. 

5.  What  confidence  do  we  have  that  the  MAV  has  been  met  or  exceeded? 

6.  What  confidence  do  we  have  that  the  SV  has  not  beer>  obtained? 

7.  Does  the  statistical  evidence  suggest  that  the  reliability  is  satis¬ 
factory,  or  not? 

Solutions 


Because  Case  Study  7-6  gave  no  evidence  of  trend,  exponential  data  analysis 
procedures  will  be  used.  Note  that  they  are  all  based  on  test  exposure,  T  = 
22,159  kilometers,  and  number  of  failures,  r  =  7.  Actual  odometer  readings  at 
failure  need  not  be  considered,  except  to  note  that  the  test  exposure  is 
"time"  terminated. 


1.  Point  estimates  of  0,  R(100),  and  k. 
a .  Apply  equation  7 . 6 

a  t  22  1S9 

6  =  -  =  =  3165.6  MKBOMF 

r  7 

Convert  to  mission  reliability  using  equation  7.7: 
R(x)  =  «-x/§ 

S(IOO)  =e-100/3165'6=e-°-'316  =0.969. 
Convert  to  failure  rate  using  equation  7.8: 


k  = 


1 

A 

6 


1 

3165.6 


=  0.000316  failures  per  km. 


b. 


Use  a  reliability  computer. 
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The  next  two  figures  illustrate  the  two-step  solution  procedure  for  part  1 
using  the  reliability  computer. 


1.  «E  ARE  LOOKING  FOR  A  POINT  ESTIMATE 

2.  NUMBER  OF  FAILURES  =  7 

3  "TIMEm=  22.159  THOUSAND  KM 

4  PT.  ESTIMATE  OF  "MT8F"  «  3.2  THOUSAND  MKBOMF 

5.  PT.  ESTIMATE  OF  FAILURE  RATE  *  .31  FAILURES  PER  THOU' 


w® 


ulitnhni!  i.lili/°,  , 


ik«w 
Kmm 

It  rri 


P 


DIRECTIONS: 

t*  im  Mw  tim  atM*  raa«M  im») 

t.  Se<  fhweeir  W  f Mure.  w  the  hr 

I.  Set  MhM  ee  am  ewe.  Tea  Twee  ee 
fteTfeCleete 

1  Weed  Mw»  Tlw»  ielwnee  Fs^was  MTSf 


ewer  Mm  lie  Twee  ee  M  Scale, 
t  Be  eel  M  career  Me  ea  fees 


WltUW  BATi 

HRPHIil 111  I  UV 


RELIABILITY 

•  '  I'l'  1 1  a 


alvm//,  .  REUABIUTY  ^  4 


P'V 


NOTE-  THE  "REUABIUTY  COMPUTER"  SHOWN  IN  THIS  ILLUSTRATION  CAN  BE  PURCHASED  FROM 
— '  TECHNICAL  AND  ENGINEERING  AIDS  FOR  MANAGEMENT,  BOX  28  TAMWORTH,  N,  H.,  056B6 


6.  PT.  ESTIMATE  OF  aMTBF<>«  3,200  MKBOMF 

7.  *TIME* »  MISSION  DURATION  -  100  KM 

8.  PI  ESTIMATE  OF  RELIABILITY  ft  .969 


ESTIMATE 


DIRECTIONS; 

*•  m  mm*  m  imm.  Mana  (wnn 
<  SM  Nga*ar  at  FMurM  i«  Hw  naiHan  Iw 

-  -  -  |  --.l 

1  MtMKVWIKMlMtkm 
ihoTlMt  taa4 

*  MUMflwlMHirillwaUW 
IX  Mura  Ma  m4fc  KXrtM 
TiMMuUn 

4.  Aaaala  ImFkiX  haMMa  raraHnr 
MWNiW  flfttaMtf  yftMl  MM  Kc#ttM9  If 

•w  HWw  r«M  a*  Tit*  tula.  > 
t  Da  aat  h*  an*  ahp  an  fwa  / 

ft*  k(4  failaiibk,  U  ■  ■  —  -J -  / 

Maw.  Miaaiaa  r«a  MM  ha  Uu  / 
IhaaTaalTMa  f  , 

Ta  aaaaart  Mura  iau  ta  /  ^ 

%  aar  iom  haura,  uu  *>a  X 
lhaa  aada  aa  aa  haw  aaata 
aa4  nuMata  tatora  raia 
aaaarar  hy  Wl.  > 
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2.  An  80%  confidence  interval  for  6  and  reliability  for  a  100-kilometer 
mission  R(100). 

a.  Using  Table  8,  Appendix  B  we  obtain  the  confidence  limit  multipliers 
for  the  case  of  7  failures  and  80%  confidence  interval,  i.e.,  90% 
upper  and  90%  lower  confidence  limits.  These  multipliers  are  0.665 
and  2.797  for  the  90%  lower  and  upper  confidence  limits, 
respectively.  Note  we  use  Table  8a  because  this  is  a  kilometers 
(i.e.,  time)  terminated  test. 

6L  =  multiplier  (6)  --  (0.595) (3165.6)  =  1883.5  MKBOMF 

Oy  =  multiplier  (6)  =  (1.797) (3165.6)  =  5688.6  MKBOMF 

We  are  therefore  80%  confident  that 
1883.5  <  6  <  5688.6  MKBOMF 

b.  Using  inequality  7.9a,  we  find,  for  a  =  0.20, 

2T_<0<  2T_ 

2  —  —  2 
Xa/2,2r+2  xl-a/2,2r 

2(22,159)  .  A  .  2(22,159) 

2  —  °  —  2 
X0. 10,16  X0.90,14 

2 

Using  Appendix  B,  Table  5  for  the  appropriate  x  values,  we  have 

44.318  „  p  ^  44,318 
23^55  -  6  -  ~7/79~ 

We  are  80%  confident  that 

1881.9  <  0  <  5689.0  MKBOMF 

In  other  words,  we  are  reasonably  sure  that  the  MKBOMF  is  not  less 
than  1881.9,  nor  greater  than  5689.0. 

c.  Converting  to  mission  reliability  using  inequality  7.16,  we  find 

<  R(x)  <  e'x/6U 

e-100/lS81.9  ,  R(100)  £  e-100/5689.0 

We  are  80%  confident  that 
0.948  <  R(100)  <  0.983 
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Commentary 

The  reliability  computer  cannot  be  used  for  confidence  intervals  since  it  does 
not  nave  a  capability  for  upper  limits. 

3.  An  80%  lower  confidence  limit  for  0. 

a.  Use  Table  8a,  Appendix  B  to  find  the  multiplier  for  an  80%  lower 
confidence  limit  with  7  failures. 

0L  *  multiplier  (6)  =  iO. 684) (3165.6)  «  2165.3  MKBOMF 

Therefore,  we  are  80%  confident  that 
6  >  2165.3  MKBOMF 

b.  Using  inequality  7.10a,  we  find 


6  > 


2T 


-  V2 

, 2r+2 


6  >  2-(2-2.159) 

“  v2 

*0.20,16 

Using  Appendix  B,  Table  5  for  the  x2  value,  we  have 
n  .  44,318 

9  i  20 fsr 

We  are  80%  confident  that 
0  >  2165.0  MKBOMF 

Using  a  reliability  computer,  we  find 


l 
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‘itl  f  ±  I 


Wiiu 


-  L  CONFIDENCE  LEVEL  *  SO% 

.  2.  NUMBER  OF  FAILURES  >  7 
.  3.  “TIME"*  22.159  THOUSAND  KM 
•  4.  LOWER  LIMIT  tt  21.7  THOUSAND  KM 


An  80%  upper  confidence  limit  for  0. 

a.  Use  Table  8a,  Appendix  B  to  find  the  multiplier  for  an  80%  upper 
confidence  limit  with  7  failures. 

Oy  =  multiplier  (0)  =  (1.<»79)(3165.6)  =  4681.9  MKBOMF 

Therefore,  we  are  80%  confident  that 

6  <  4681.9  MKBOMF 

b.  Using  inequality  7.11,  we  find 


x2 

M-a, 2r 


2 


x0. 80,14 


Using  Appendix  B,  Table  5  for  the  %  value,  we  find 
n  .  44,318 

6  i -for 

We  are  80%  confident  that 
6  <  4679.8  MKBOMF 


Continental 


The  reliability  computer  does  not  h&'je  a  capability  for  upper  limits. 

5.  What  confidence  do  we  have  that  0  >  2,000? 

a.  Using  equation  7.12a,  we  find 

2  2T 

*a,2r+2  ~  0^ 

s  2(22 , 159)  ^ 2  jkq 

xa,l6  2000 

Searching  Appendix  B,  Table  5  in  the  row  labeled  16  for  a  value  of 
22.159,  we  find  va?  >  "  of  20.47  and  23.55.  Interpolating,  we  obtain 
a  £  0.14.  Confix cc  .e  is  100(l-Cf)%  £  ^00(1-0. 14)%.  We  are  approxi¬ 
mately  86%  confident  that 


0  >  2000  MKBOMF 
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b.  Using  equation  7.13a,  we  find 


r  (T/6T)k  e"(T/V 

a  =  I  - L 


k=0 


k( 


W«i  *  ToSF  *  11 -0795 


r  a  7 


-  Ol.0795)°e"11,0795  .  qi,0795)V11,0795 


1 


1 


,  (11.07957  V11,11795  ,  fU.0795iy110795 


.  (11.0795)V11  0795  .  (11.0795>V11  0795 

*  Ml  ■  - 


24 


120 


.  (11.0795)V11-0795  .  (11.0795)V11-0795 

'  A  *  r  a<  A 


720 


5040 


a  0.0000  +  0.0002  +  0.0009  ♦  0.0035  +  0.0097 
+  0.0215  +  0.0396  +  0.0627 


a  0.1381 


We  are  86.2%  confident  that 


6  >  2000  MKBOMF 


6.  What  confidence  do  we  have  that  0  <  4,0007 


The  confidence  that  6  <  4,000  is  the  same  as  the  confidence  that  0  <  4,000. 
The  fortser  statenent  is  easier  to  interpret,  although  the  latter  is  the  usual 
expression. 

a.  Using  equation  7.14a,  we  find 


2  2T 

Va,2r  "  0^ 


v2  _  2(22,159) 
xl-a,14  4000 


11.08 


Searching  Appendix  B,  Table  5  in  the  row  labeled  14  for  a  value  of 
11.08,  ve  find  values  of  10.16  and  13.34.  Interpolating,  we  obtain 
1-a  3  0.68.  Confidence  is  100(l-a)%  3  100(0.68)%.  We  are  approxi¬ 
mately  68%  confident  that 

6  <  4000  MKB0MF 

b.  Using  equation  7.15,  we  obtain 

r-1  (T/6,,)k  e’^V 

I  - ^ - 

k=0 

I/6u  “  To®4  ■  5  • 53975 


r-1  =  6 


1-a  =  (5.53975)V5>53975  + 


(5. 53975)  V5  *' 53975 


(5.53975)2e"5‘53975 


(5.53975)V5,53975 


+  (5. 53975  )V5, 53975 
24 


j5e-5. 53975 


(5.53975)V5,53975 


=  0.0039  +  0.0218  +  0.0603  +  0.1113  +  0.1541  +  0.1708 
+  0.1577 
=  0.6798 


We  are  68%  confident  that 
6  <  4000  MKBOMF 


7.  Does  the  reliability  appear  satisfactory? 

We  are  86%  confident  that  the  user's  needs  have  been  net,  but  only  68%  con¬ 
fident  that  contractual  obligations  were  uot  net.  There  is  stronger  evidence 
that  the  reliability  is  satisfactory  than  not.  If  sumy  more  failures  were 
experienced,  we  would  have  low  confidence  that  the  user's  needs  were  net,  and 
we  would  also  have  higher  confidence  that  the  contractual  obligations  were  not 
net,  suggesting  that  the  reliability  is  not  satisfactory  from  both 
standpoints . 


CASE  STUDY  NO.  7-8 


Background 

An  avionics  system  has  the  following  reliability  requirements:  the  minimum 
acceptable  value  (MAV)  =  150  hra.  MTRF,  and  the  specified  value  (SV)  =  450 
hra.  MTBF.  Three  of  these  systems  were  tested  for  100  hours  (each)  under  test 
conditions  that  closely  duplicated  the  expected  operational  environment.  No 
failures  were  observed  during  this  test. 

Determine 


An  80%  lower  confidence  limit  for  MTBF,  and  the  confidence  that  the  MAV  has 
been  attained. 

Commentary 

The  case  of  a  test  with  zero  failures  has  some  interesting  features.  With  no 
failures,  there  is  no  way  to  determine  the  type  of  failure  pattern.  If  we 
have  some  assurance  that  the  equipment  will  not  degrade  as  it  ages,  we  can 
make  a  constant  failure  rate  assumption,  which,  in  turn,  permits  an  ex¬ 
ponential  data  analysis. 

If  we  attempt  to  obtain  a  point  estimate  of  6,  we  get: 

8  =  -  s  ~  =  indeterminate 
r  o 

Similarly,  the  upper  limit  is  indeterminate.  We  can,  however,  obtain  lower 
confidence  limits. 

Solutions 


1.  80%  lower  confidence  limit  for  8. 

a!  Note  that  the  technique  of  using  the  multipliers  from  Table  8, 
Appendix  B,  cannot  be  used  for  the  case  of  zero  failures. 

b.  Using  inequality  7.10a,  we  find 

e>^3_ 

xa,2r+2 

We  have  .In  this  case,  T  =  300,  a  =  0.2  and  r  *  0,  so 

0  >  2|3 fifii 
X0 . 2 , 2 
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Using  Appendix  B,  Table  5  for  the  x  value,  we  find 


6 


600 

3.22 


We  are  80%  confident  that  8  >  186.3  hrs  MTBF 
b.  Using  a  reliability  computer,  we  find 


-  I.  CONFIDENCE  LEVEL  *  80% 

-  2.  NUMBER  OF  FAILURES  *  0 

-  3.  TIME  *  300  HOURS 

| -  4.  LOWER  LIMIT  tt  186  HOURS 


2.  Confidence  that  8  >  ISO 


As  noted  in  pact  1  of  this  problem,  Table  8  cannot  be  used  for  the 
case  of  zero  failures. 

Using  equation  7.12a,  we  find 

¥2  _  2T 

*a,2r+2  “  8^ 


2  a  2j3002  s  4  0 
xa,2  150  4,u 

Searching  Appendix  B,  Table  S  in  the  row  labeled  2  for  a  value  of 
4.0,  we  find  values  of  3.22  and  4.60.  Interpolating,  we  obtain  a  ~ 
.13.  Confidence  is  I00(l-a)%  -  100(1-0.13)%.  We  are  approximately 
87%  confident  that 


8  >  150  hrs  MTBF 
Using  equation  7.13a,  we  find 

r  CT/eT)k  «-(T/V 

«=  I  — - 

k-0 

For  r  -0,  this  simplifies  to 

« =  ,-n> 


In  this  case, 


«  =  ,-«00/150)  .  e"2  =0.135 


Ve  are  86.5%  confident  that 
8  >  150  hrs  MTBF 


Mk  ^ 


WP 
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CASE  STUDY  NO.  7-9 


Background 

A  system  is  being  tested  using  test  plan  XIIC  from  Appendix  B,  Table  6.  The 
upper  test  value  (SV)  is  100  hours  MTBF,  and  the  lower  test  value  (MAV)  is  50 
hours  MTBF.  The  required  test  duration  is  940  hours,  and  14  failures  are 
rejectable.  The  source  of  data  for  this  test  plan  is  not  relevant  for  this 
case  study,  but  is  presented  here  for  future  reference.  Chapter  8  contains 
detailed  discussions  on  the  formation  and  use  of  this  and  other  test  plans. 

The  seventh  failure  has  just  occurred  after  only  57  hours  of  test  exposure. 
Because  of  the  excessive  number  of  failures,  an  evaluation  is  to  be  done  at 
this  point  in  the  test.  Preliminary  analysis  of  the  data  showed  no  evidence 
of  trend,  i.e.,  failure  rate  appeared  constant. 

Determine 


1.  Point  estimate  of  MTBF. 

2.  80%  confidence  interval  for  MTBF. 

3.  80%  lower  confidence  limit  for  MTBF. 

4.  80%  upper  confidence  limit  for  MTBF. 

5.  What  confidence  do  we  have  that  the  lower  test  value  has  been  met  or 
exceeded? 

6.  What  confidence  do  we  have  that  the  upper  test  value  has  not  been  at¬ 
tained? 

7.  Does  the  statistical  evidence  suggest  that  the  reliability  is  satis¬ 
factory  or  not? 

Commentary 

Because  an  evaluation  is  being  made  at  this  point  based  on  what  was  observed, 
we  do  not  have  a  legitimate  random  sample.  The  true  risks  in  making  decisions 
based  on  such  an  analysis  are  difficult  to  determine.  They  are,  in  fact, 
substantially  higher  than  the  ones  associated  with  the  original  plan.  Conse¬ 
quently,  the  following  analyses  are  all  somewhat  pessimistically  biased. 

Solutions 

I 

Since  the  seventh  failure  has  just  occurred,  this  is  failure  terminated  data. 
1.  Point  estimate  of  6. 
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Applying  equation  7.6,  we  obtain 

6  =  |  =8.14  hrs  MTBF 

An  80%  confidence  interval  for  6. 

a.  Use  Table  8b,  Appendix  B  to  obtain  the  confidence  limit  multiplier 
for  the  case  of  7  failures  and  80%  confidence  interval,  i.e.,  90% 
upper  and  lover  confidence  limits.  Note  ve  are  using  Table  8b 
because  the  test  is  failure  terminated. 

Oy  =  multiplier  (6)  =  (2.797)(8.14)  *  14.63  hrs.  MTBF 

0L  =  multiplier  (9)  =  (0.665) (8. 14)  =  5.41  hrs.  MTBF 

We  are  therefore  80%  confident  that 

5.41  <  9  <  14.63  hrs.  MTBF 
Using  inequality  7.9b,  we  find,  for  a  =  0.20, 


b. 


2T 


<  9  < 


2T 


2  -  -  2 
xa/2,2r  xl-a/2,2r 

o 

Using  Appendix  B,  Table  5  for  X  values: 


114  <e<  114 


21.07  -  -  7.79 

We  are  80%  confident  that 

5.41  <  6  <  14.63  hrs  MTBF 
An  80%  lower  confidence  limit  for  9. 

a.  Use  Table  8b,  Appendix  B  to  find  the  multiplier  for  an  80%  lower 
confidence  limit  with  7  failures. 

9l  =  multiplier  (9)  =  (0.771) (8. 14)  =  6.28  hrs.  MTBF 

Therefore,  we  are  80%  confident  that 
9  >  6.28  hrs.  MTBF 

b.  Using  inequality  7.10b,  we  find 
2T 


9  > 


-v2 

*a,2r 
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Using  Appendix  B,  Table  5  for  the  x  value,  we  fine' 

ft  >  114 

6  -  18.15 

We  are  80%  confident  that 
6  >  6.28  hrs  MTBF 

An  80%  upper  confidence  limit  for  6. 

a.  Use  Table  8b,  Appendix  B  to  find  the  multiplier  for  an  80%  upper 
confidence  limit  with  7  failures. 

Oy  =  multiplier  (§)  =  (1.479) (8. 14)  =  12.04  hrs.  MTBF 

Therefore,  we  are  80%  confident  that 
8  <  12.04  hrs.  MTBF 

b.  Using  inequality  7.11,  we  find 

e<-^~ 

xl-a,2r 

o 

Using  Appendix  B,  Table  5  for  the  x  value,  we  obtain 

B  <  11* 

e  <9757 

We  are  80%  confident  that 
8  <  12.04  hrs  MTBF 

What  confidence  do  we  have  that  0  >  50? 


a.  Using  equation  7.12b,  we  find 


=  2T/6L 


xa,l4  s  2<57>/50  s  2*28 


Searching  Appendix  B,  Table  5  in  the  row  labeled  14  for  a  value  of 
2.28,  we  find  that  wc  are  beyond  the  end  of  the  table,  and  a  > 
0.995.  The  confidence  is  100(l-a)%,  100(1“'' .995)%,  or  less  than 
0.5%. 

We  are  less  than  0.5%  confident  that 


8  >  50  hrs  MTBF 
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b.  Using  equation  7.13b,  we  find 


a 


r-1  (T/6r) 

1  - — 

k=0 


k  e*(T/8L) 
__ 


where 


6. 


T/6l  =  57/50  =  1.14  and  r  -  1  =  7  -  1  =  6. 

Solving  equation  7.13b,  we  find  a  =  0.9998. 

We  are  0.02%  confident  that  0  >  50  hours  MTBF. 
What  confidence  do  we  have  that  6  <  100? 
a.  Using  equation  7.14,  we  find 


v2  -  2(57)  _ 
*l-ot,l4  "  100  ~ 


1.14 


Searching  Appendix  B,  Table  5  in  the  row  labeled  14  for  a  value  of 
1.14,  we  find  that  we  are  well  beyond  the  end  of  the  table,  and  1-a 
=  1.0.  The  confidence  is  100(l-a)%,  100(1.0)%,  or  essentially  100%. 

We  are  essentially  100%  confident  that 

0  <  100  hrs  MTBF 

b.  Using  equation  7.15,  we  find 

r-1  (T/6n)k  e'(T/V 
!■“=  *  - r-n - 

k=0  K1 

where 


T/6U  =  0.57  and  r  -  1  =  7  -  1  =  6 


Solving  equation  7.15,  we  find  1-a  =  0.99999. 
We  are  essentially  100%  confident  that 
6  <  100  hrs  MTBF 


7.  Does  the  reliability  appear  satisfactory? 

Since  we  have  essentially  0%  confidence  that  the  lower  test  value  is  met  or 
exceeded,  and  since  we  have  essentially  100%  confidence  that  the  upper  test 
value  is  not  met,  there  is  overwhelming  evidence  that  the  reliability  is  not 
satisfactory,  even  taking  into  consideration  the  fact  that  the  analysis  may  be 
somewhat  pessimistically  biased. 


4 
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In  this  case,  the  evidence  is  so  strong  that  we  can  even  state  that  we  are 
99.9S%  confident  that  0  <  50  hrs  MTBF,  though,  ordinarily,  upper  limit  state¬ 
ments  are  associated  with  the  upper  test  value,  and  lower  limit  statements  are 
associated  with  the  lower  test  value. 

Commentary 

Test  plan  XIIC  from  Appendix  B,  Table  6  required  a  test  duration  of  940  hours 
to  achieve  true  producer's  and  consumer's  risks  of  0.096  and  0.106,  re¬ 
spectively.  Since  the  system  appears  to  be  "failing  miserably,"  the  user  has 
chosen  to. stop  testing  after  57  hours.  No  doubt  this  is  a  wise  decision  from 
an  economic  standpoint.  However,  the  user  should  be  fully  cognizant  that  the 
risks  associated  with  his  abnormally  terminated  test  are  not  0.096  and  0.106, 
nor  are  they  the  ones  advertised  in  the  preceding  computations.  The  calcula¬ 
tion  of  the  true  risks  is  well  beyond  the  scope  of  this  work. 
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CHAPTER  8 


RELIABILITY  TEST  PLANNING 


INTRODUCTION 

This  chapter  presents  the  techniques  for  determining  the  amount  of  test  expo¬ 
sure  required  to  satisfy  previously  established  program  reliability  require¬ 
ments.  The  reader  will  note  that  Chapter  7  addresses  the  topic  of  reliability 
data  analysis.  There,  we  assumed  that  the  test  data  had  already  been 
gathered.  We  then  used  the  available  data  to  determine  point  estimates  for 
reliability  parameters  and  to  stipulate  the  uncertainty  associated  with  these 
estimates . 

Chapter  8  presents  techniques  for  designing  test  plans  which  can  verify  that 
previously  specified  reliability  requirements  have  been  achieved.  We  realize, 
of  course,  that  the  required  test  exposure  and/or  sample  siz  may  exceed  the 
available  resources.  In  such  cases,  alternative  test  plans,  consistent  with 
program  constraints,  must  be  developed.  In  this  chapter,  we  also  present 
methods  which  make  it  possible  to  clearly  identify  the  inherent  risks  associ¬ 
ated  with  a  limited  test  program. 

PRIMARY  TEST  DESIGN  PARAMETERS 

Upper  and  Lower  Test  Values 

Two  values  of  system  reliability  are  of  particular  importance  in  the  design  of 
a  reliability  test  plan.  These  are  referred  to  as  the  upper  test  and  lower 
test  values.  In  some  cases,  only  a  single  value  is  initially  apparent,  the 
second  value  being  only  implied.  These  two  values  and  the  risks  associated 
with  them  determine  the  type  and  magnitude  of  testing  required. 

The  upper  test  value  is  the  hoped  for  value  of  the  reliability  measure.  An 
upper  test  MTBF  is  symbolized  as  0q,  and  an  upper  test  reliability  is  symbol¬ 
ized  as  Rq.  A  test  plan  is  designed  so  that  test  systems  whose  true  reli¬ 
ability  parameters  exceed  0^  and  Rq  will,  with  high  probability,  perform 
during  the  test  in  such  a  way  as  to  be  "accepted." 

The  lower  test  value  is  commonly  interpreted  in  two  different  ways  that  may 
initially  appear  contradictory.  One  interpretation  is  that  this  lower  value 
of  the  reliability  measure  represents  a  rejection  limit.  The  other  interpre¬ 
tation  is  that  this  value  is  minimally  acceptable.  The  apparent  conflict  is 
resolved  by  viewing  the  lower  test  value  as  the  fine  line  between  the  best 
rejectable  value  and  the  worst  acceptable  value.  A  lower  test  MTBF  is  symbol¬ 
ized  as  0J,  and  a  lower  test  reliability  is  symbolized  as  R^.  Systems  whose 

true  reliability  parameters  having  values  less  than  0^  and  R^  will,  with  high 

probability,  perform  in  such  a  way  as  to  be  "rejected." 
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The  upper  and  lower  test  values  serve  to  divide  the  reliability,  or  MTBF, 
scale  into  three  distinct  regions  as  shown  in  Figure  8-1.  Note  that  the 
region  between  R^  and  Rq  is  neither  bad  enough  to  demand  rejection  nor  is  it 

good  enough  to  demand  acceptance.  This  region  is  necessary  since  we  will 
never  precisely  know  the  true  reliability  of  the  system. 


FIGURE  8-1  REGIONS  DEFINED  BY  R  AND  R, 

0  1 


0  R.  Ro 

TRUE  RELIABILITY,  R 

♦MAY  BE  DIFFERENT  DEFENDING  ON  ACTUAL  MATURITY. 
SEE  FOLLOWING  PARAGRAPH. 


The  user's  reliability  requirement  should  be  stated  as  a  minimum  acceptable 
value  (MAV);  that  is,  the  worst  level  of  reliability  that  the  user  can  tol¬ 
erate  and  accept.  The  contractually  specified  value  (SV)  is  a  value  somewhat 
higher  than  the  MAV.  For  reliability  qualification  tests  prior  to  production, 
the  lower  test  value  is  the  MAV,  and  the  upper  test  value  is  the  SV.  Earlier 
in  the  development  process,  fixed  configuration  tests  may  be  conducted  to 
demonstrate  the  attainment  of  lower  levels  of  reliability  at  specified  mile¬ 
stones.  In  such  cases,  upper  test  and  lower  test  values  should  be  consistent 
with  the  stage  of  the  development  process. 

In  the  above  paragraphs,  we  have  been  discussing  population  parameter  values 
only.  These  values  are  never  known  with  absolute  certainty,  so  we  are  forced 
to  base  an  evaluation  of  system  performance  characteristics  on  sample  data. 
Let  us  conclude  this  section  with  a  discussion  of  sample  reliability  values 
and  how  we  can  interpret  them  to  aid  us  in  making  our  system  reliability 
assessment. 

One  objective  of  this  chapter  is  the  determination  of  an  accept/reject 
criterion  for  a  test  to  be  conducted.  As  an  example,  consider  the  value  R^  in 

Figure  8-2  below.  The  term  R^,  is  that  value  of  the  sample  reliability  which 

corresponds  to  the  maximum  allowable  number  of  failures  that  can  occur  during 
testing  and  still  result  in  acceptance  of  the  system. 

If  we  test  our  determined  number  of  articles  and  find  that  Rsamp^e  Is  larger 
than  Rt,  then  we  accept  the  system  because  there  is  high  probability  that  the 
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FIGURE  8-2  SAMPLE  RELIABILITY  RANGE 


sample  system(s)  come  from  a  population  of  systems  whose  true  reliability  R 
exceeds  Rp,  the  minimum  acceptable  value  (MAV)  (see  Figure  8-1)  for  this  test. 


Note  that  when  Rgamp^e  is  larger  than  Rp,  we  have  confidence  that  the  true 

reliability  exceeds  the  MAV.  We  should  not  interpret  this  result  as  an  indi¬ 
cation  that  the  contractor  has  met  the  SV.  Further,  if  R  ,  is  smaller 

'  sample 

than  Rp,  we  will  reject  the  system  because  there  is  high  probability  that  the 


sample  system(s)  come  from  a  population  whose  true  reliability  R  is  lower  than 
Rq,  the  SV  for  this  test.  Note  that  when  Rsamp^e  is  smaller  than  Rp,  we  have 

confidence  that  the  true  reliability  falls  short  of  the  SV.  We  should  not 
interpret  this  result  as  an  indication  that  the  MAV  has  not  been  met,  but 
rather  that  the  MAV  has  not  been  demonstrated  at  a  sufficiently  high  level  of 
confidence . 


Consumer's  Risk 


and  Producer's  Risk 


The  consumer's  risk  (p)  is  the  probability  of  accepting  the  system  if  the  true 
value  of  the  system  reliability  measure  is  less  than  the  lower  test  value.  It 
can  be  interpreted  in  the  following  ways: 


1.  p  represents  the  maximum  risk  that  the  true  value  of  the  reliability 
measure  is,  in  fact,  less  than  the  lower  test  value. 


2.  From  an  alternative  viewpoint,  if  the  acceptance  criterion  is  met,  there 
will  be  at  least  100(l-p)%  confidence  that  the  true  value  of  the  reli¬ 
ability  measure  equals  or  exceeds  the  lower  test  value. 

The  producer's  risk  (a)  is  the  probability  of  rejection  if  the  true  value  of 
the  reliability  measure  is  greater  than  the  upper  test  value.  It  can  be 
interpreted  in  the  following  ways: 

1.  The  probability  of  acceptance  will  be  at  least  (1-a)  if  the  upper  test 
value  is,  in  fact,  met  or  exceeded. 

2.  From  an  alternative  viewpoint,  if  there  is  a  rejection  decision,  there 
will  be  at  least  100(l-a)%  confidence  that  the  true  value  of  the  reli¬ 
ability  measure  is  less  than  the  upper  test  value. 

Case  study  8-1  illustrates  the  relationship  between  a  and  p. 


Pre-  and  Post-Test  Risk  Considerations 

-  I  M  . . -  . — -  ■■ 

Before  proceeding  on  with  the  application  of  the  consumer’s  and  producer's 
risk  concept .  it  is  important  to  understand  the  contrast  that  exists  between 
pre-and  post-test  risks. 

The  a  and  £  risks  represent  uncertainties  that  exist  in  the  test  planning  or 
pre-data  environment  discussed  in  this  chapter.  Once  data  has  been  gathered 
and  we  have  accepted  or  rejected  the  system,  we  find  that  the  risk  environment 
is  altered.  For  example,  we  take  a  sample  and  decide  to  accept  the  system. 
At  this  point  the  producer's  risk  is  eliminated;  the  consumer's  risk  remains 
but  is  less  than  the  maximum  that  would  exist  had  the  sample  reliability, 

Rsample  *  beeri  exactly  e^ual  t0  V 

If,  on  the  other  hand,  Rsampie  is  less  than  R^,,  i.e.,.  we  reject  the  system,  we 

find  that  the  consumer's  risk  is  eliminated  since  there  is  no  risk  of  ac¬ 
cepting  a  bad  system.  Likewise,  the  producer' &  risk  is  less  than  the  maximum 
that  would  exist  had  the  sample  reliability  ®gaBpj.e  been  exactly  equal  to  R^,. 

In  this  chapter,  we  are  concerned  with  pre-test  risks.  We  determine  the 
maximum  a  and  p  risks  and  then  calculate  the  required  test  exposure  and  ac¬ 
ceptable  number  of  failures  which  will  limit  our  risk  to  the  maximum  levels. 


Four  values  specify  the  plan  for  a  binomial  test.  They  are: 

-  the  specified  or  desired  proportion  of  failures  (pQ), 

-  the  maximum  acceptable  proportion  of  failures  (pj), 

-  the  consumer's  risk  (0), 

-  the  producer's  risk  (a). 

The  test  plan  itself  consists  of  a  sample  size  (n)  and  an  acceptance  criterion 
(c) .  The  value  c  represents  the  maximum  number  of  failures  which  still  re¬ 
sults  in  acceptance  of  the  system.  It  is  usually  not  possible  to  construct  a 
plan  which  attains  the  exact  values  of  a  and  p.  There  are  however  plans  which 
attain  risks  which  do  not  exceed  a  and  p.  We  shall  present  methods  for  de¬ 
termining  these  types  of  plans,  though  in  a  real  world  situation,  the  user  and 
producer  may  trade  off  some  protection  to  achieve  other  goals. 

The  following  paragraphs  present  exact  and  approximate  procedures  to  be  used 
in  planning  a  Discrete  Time-Binomial  Model  test  program.  The  "exact  pro¬ 
cedure"  presents  the  equations  used  to  determine  the  two  values  required  to 
specify  a  binomial  test  plan.  These  equations  are  presented  here  for  the  sake 
of  completeness.  The  "approximate  solution"  procedure,  which  makes  use  of  the 
binomial  tables  to  simplify  the  procedure,  is  intended  for  use  by  our  readers. 
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Exact  Solution  Procedure 

The  exact  procedure  for  determining  test  plans  for  the  four  values  listed 
above  is  to  solve  the  following  two  inequalities  simultaneously  for  c  and  n. 


1  (! 

:) u-P!)"-*  < » 

(8.1) 

k-0  V* 

n  j 

1  (£ 
k=c+r* 

)  pj  a-p0>Q~k  <  ■ 

(8.2) 

There  are  an  infinite  number  of  solutions  to  this  pair  of  inequalities.  The 
plans  of  interest  are,  of  course,  those  which  minimize  the  sample  size  (n) 
required.  Solving  inequalities  8.1  and  8.2  directly  is  next  to  impossible 
without  the  aid  of  a  computer.  MIL-STD-105D  contains  numerous  binomial  test 
plans  which  may  be  used  for  reliability  applications.  We  should  point  out 
that  the  user  unfamiliar  with  this  document  will  find  it  difficult  to  inter¬ 
pret,  thus  we  present  the  following  procedures. 

Approximate  Solution  Procedures 

The  following  so-called  approximate  procedures  utilize  the  normal  and  Poisson 
distributions  to  obtain  approximate  solutions  to  equations  8.1  and  8.2  and 
thereby  estimate  values  of  the  sample  size  (n)  and  the  acceptance  criterion 
(c).  After  approximate  values  for  these  parameters  have  been  obtained,  we  may 
then  use  the  values  in  conjunction  with  the  binomial  tables  (Appendix  B)  and 
the  previously  selected  and  fixed  values  of  a  and  0  to  "fine  tune"  the  ap¬ 
proximate  values  of  n  and  c. 

Test  Planning  Using  Normal  Approximation.  The  normal  distribution  provides 
good  approximations  for  solving  inequalities  8.1  and  8.2,  especially  for 
moderate  values  of  p  (0.1  <  p  <  0.9).  Using  this  information,  we  obtain  the 
approximate  solutions  for  n  and  c  as  follows. 

_  zoV>'0)  +  *  2Vb 

n  |  (fj  ,  J  J 

(prp0>2 


c  ~  za  Vnp0(l-P 0)  +  npQ  “  0.5  .  (8.4) 

Generally,  the  values  computed  using  equations  8.3  and  8.4  are  good  approxima¬ 
tions  for  the  test  planner.  When  p^  and  p1  are  veiy  small  (less  than  0,05), 

the  procedure  is  not  recommended.  Fine-tuning  of  the  test  plan  may  still  re¬ 
quire  solving  the  original  inequalities  or  some  bargaining  with  user  and/or 
producer. 

As  an  example,  suppose  that  the  minimum  acceptable  reliability  of  a  system  is 
0.85  (pj  -  0.15),  while  the  contractually  specified  reliability  is  0.95  (pQ  = 
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0.05).  Consumer  and  producer  risks  of  0.11  are  required,  i.e.,  a  =  P  =  0.11. 
For  a  =  0.11,  z^  =  1.225  aud  *or  p  =  0.11,  z^  =  1.225.  (These  valuta  of  za 

and  Zp  are  obtained  from  Appendix  B,  Table  2.)  Using  the  normal  approxima¬ 
tion,  we  have 

n  =  {(1.225)^(0.05-0.002.5)  +  (1.225)2(0. 15-0.0225) 

+  2(1.225)2V<;07o"5y(07i5)  (0.95)  (0.8571/(0. 15-0. 05)2 
=  49.6 


and 


c 


(41.-.  6)  (0.05) 


0.5 


=  3.9  . 


The  values  of  n  =  49.6  and  c  =  3.9  are  initial  approximations.  In  order  to 
fine  tune  the  test  plan,  we  round  these  values  to  n  =  50  and  c  =  4  and  use  the 
binomial  tables  (Appendix  B,  Table  1).  For  an  n  of  50  and  a  c  of  4,  the 
probability  of  c  or  fewer  failures  when  p  =  Pj  =  0.15  is  0.1121.  In  addition, 

the  probability  of  c  or  fewer  failures  when  p  =  pQ  =  0.05  is  0.8964.  Thus, 

for  the  test  using  a  sample  size  of  50  with  a  maximum  acceptable  number  of 
failures  of  4,  the  producer's  risk  0=  1  -  0.8964  =  0.1036,  and  the  consumer's 
risk  p  a  0.1121.  Note  that  these  values  were  obtained  directly  from  Appendix 
B,  Table  1.  It  would,  however,  have  been  difficult  at  best  to  decide  where  to 
begin  looking  in  the  binomial  tables  without  having  first  used  the  normal 
approximation  for  guidance. 

Test  Planning  Using  Poisson  Approximation.  The  Poisson  distribution  also 
provides  reasonable  approximations  to  inequalities  8.1  and  8.2.  All  this 
amounts  to  is  substituting  np  for  At  or  t/8  in  the  Poisson  distribution  equa¬ 
tion.  Consequently,  approximate  values  for  n  and  c  are  obtained  by  solving 
the  following  inequalities. 


c  (np  )ke“Qpl 

2  - i- -  <  p.  (8.5) 

k=0 


c  (npo)ke“ap0 

2  - HTi - >  1  -  a-  (8.6) 

k=0  K! 


Standard  test  plans  and  procedures  for  the  Poisson  (exponential)  are  readily 
available  and  may  be  used  in  lieu  of  solving  inequalities  8.5  and  8.6.  This 
subject  is  discussed  in  the  "Sources  of  Exponential  Test  Plans"  section  of 
this  chapter.  To  use  these  piano  in  this  context,  we  let  0Q  =  l/pQ,  = 

l/p1#  n  *  T,  and  use  the  acceptable  number  of  failures  as  given. 


Am  an  example,  suppose  that  the  minimum  acceptable  reliability  of  a  system  is 
0.9  (Pj  =  0.1)  and  the  contractually  specified  reliability  is-  6.95  (pQ  = 

0.05).  Consumer  and  producer  risks  are  to  be  20%,  i.e.,  a  =  0  -  0.20.  To  use 
the  Poisson  approximation,  we  define  8q  =  l/p^  =  1/0.05  =  20  and  6^  =  1/pj  = 

1/0.1  =  10.  The  discrimination  ratio,  8q/0j,  is  2.  Note  that  test  plan  XIVC 

in  Appendix  B,  Table  6,  has  a  and  0  risks  of  19.9%  and  21.0%,  respectively. 
This  plan  requires  a  teat  duration  T,  corresponding  to  n  for  this  example,  of 
(7. 8) (6j)  or  78,  with  five  or  fewer  failures  being  acceptable. 

The  term  "discrimination  ratio"  and  the  use  of  Appendix  B,  Table  6,  test  plans 
are  discussed  in  detail  in  the  following  section. 

Case  Studies  8-1,  8-2,  and  8-3  demonstrate  the  development  of  binomial  test 
plans  for  a  variety  of  a  and  £  values. 
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The  main  feature  of  test  planning  for  continuously  operating  systems  based  on 
the  exponential  distribution  is  the  assumption  that  the  systems  have  a  con¬ 
stant  failure  rate. 


Requirement  Interpretation 


When  the  user's  requirement  is  stated  in  terms  of  ah  MTBF,  there  is  an  impli- 
catiou  of  a  constant  failure  rate.  This  does  not  mean  that  the  system  must 
have  a  constant  failure  rate.  It  means,  instead,  that  the  need  remains  con¬ 
stant.  Figure  8-3  illustrates  that  the  user's  needs  may  be  met  during  only  a 
portion  of  the  time  during  the  life  of  a  system. 


FIGURE  8-3  USER  REQUIREMENTS  VS  SYSTEM  PERFORMANCE 


FAILURE 

RATE 


Constant  System  Failure  Rate  Assumption 

The  assumption  that  the  system  to  be  tested  has  a  constant  failure  rate  may 
not  be  a  good  one,  but  it  is  a  practical  necessity  for  determining  the  amount 
of  teating  required.  In  theory,  with  the  constant  failure  rate  assumption 
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only  the  total  test  exposure  is  important.  That  is  (in  theory),  one  system 
could  be  tested  for  the  required  test  exposure,  or  many  systems  could  be 
tested  for  a  short  time. 

In  practice,  a  test  should  be  planned  with  a  moderate  number  of  systems  on 
test  for  a  moderate  period  of  time.  This  makes  the  test  relatively  insensi¬ 
tive  to  the  constant  failure  rate  assumption.  For  example,  one  organization 
recommends  that  at  least  three  systems  be  tested  for  at  least  three  times  the 
MAV  (each) .  These  are  constraints  imbedded  in  the  required  total  test 
exposure . 

Discrimination  Ratio 


The  discrimination  ratio,  d  =  0^/ 0j,  is  a  parameter  useful  in  test  planning 

for  the  exponential  model.  (For  the  binomial  model,  it  is  necessary  to  con¬ 
sider  the  upper  and  lower  test  values,  p^  and  p^,  explicitly  along  with  the  a 

and  0  risks.)  An  interesting  feature  of  the  exponential  model  is  that  only 
the  ratio  of  the  upper  and  lower  test  values,  d  =  Oq/6^,  along  with  the  a  and 

0  risks  need  to  be  considered.  As  a  consequence,  test  plans  for  the  ex¬ 
ponential  models  address  explicitly  the  discrimination  ratio  as  a  planning 
parameter. 

Sources  of  Exponential  Test  Plans 

There  are  numerous  methods  and  references  available  for  developing  exponential 
test  plans.  Three  such  approaches  are: 

1.  MIL-STD  105D  and  MIL-HBK  108. 

2.  MIL-STD  781C  Test  Plans. 

3.  Poisson  Distribution  Equations. 

Reference  to  MIL-STD  105D  and  MIL-HBX  108  is  included  here  solely  for  the  sake 
of  completeness.  It  is  our  intention  that  the  reader  become  familiar  with 
methods  of  exponential  test  planning  using  MIL-STD  781C  and  the  Poisson  dis¬ 
tribution  equations.  These  methods  are  described  below.  All  the  necessary 
excerps  from  MIL-STD  781C  are  provided  in  Appendix  B,  Tables  6  and  7. 

MIL-STD  105D  and  MIL-HBK  108.  MIL-STD  105D  is  a  document  devoted  primarily  to 
binomial  and  Poisson  sampling  plans,  and  as  such,  is  mentioned  in  the  previous 
section.  The  Poisson  sampling  plans  may  be  used  for  continuous  time  reli¬ 
ability  tests.  MIL-HBK  108  is  devoted  to  reliability  testing  based  on  the 
exponential  distribution.  However,  it  is  limited  in  use  for  our  purposes 
because  it  describes  test  plans  for  the  situation  when  the  test  time  per  unit 
on  test  is  preset  and  the  number  of  units  is  determined.  We  iterate  here  that 
these  documents  are  difficult  to  interpret,  and  as  such,  should  only  be  used 
by  a  person  familiar  with  their  content. 

MIL-STD  781C.  The  required  excerpts  from  MIL-STD  781C  are  provided  in  Appen¬ 
dix  B,  Tables  6  and  7.  Both  tables  provide  information  which  enable  the 
reader  to  design  a  test  program  which  addresses  established  requirements.  The 
following  paragraphs  detail  the  use  of  both  tables. 
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Appendix  B,  Table  6:  Exponential  Test  Plans  for  Standard  Discrimination 
Ratios.  This  table  presents  information  which  supports  the  development  of  a 
test  plan  based  on  discrimination  ratios  of  1.5,  2.0,  and  3.0.  For  each  of 
these  discrimination  ratios,  four  test  plans  are  provided  which  attain  ap¬ 
proximate  a  and  £  risks  of  10%  and  10%,  10%  and  20%,  20%  and  20%,  and  30%  and 
30%.  Figure  8-4,  in  conjunction  with  the  following  example  problem,  il¬ 
lustrates  the  use  of  Appendix  B,  Table  6. 

FIGURE  8-4.'  HOW  TO  USE  APPENDIX  B,  TABLE  6  . 

- 1.  Identify  rows  corresponding  to  the  specified  "d". 

(If  not  in  Appendix  B,  Table  6,  use  Table  7  plans.) 

- 2.  Identify  desired  a  and  0  risks. 

- 3.  Identify  test  plan  number  (for  reference). 

— 4.  Identify  test  duration  multiplier. 

5.  Determine  total  test  time  as  6^  times  multiplier. 
r-6.  Identify  accept/reject  criteria. 


True 

Test  Decision 

Plan  Risks 


IXC*  12.0%  9.9% 

XC  10.9%  21.4% 

XIC  17.8%  22.4% 

XIIC  9.6%  10.6% 

XIVC  19.9%  21.0% 

XVC  |  - 1  9.4%  9.9%~ 

XVIC  10.9%  21.3% 

XVIIC  17.5%  19.7% 


Test 

Duration 

Discrimination  Multiplier  (M) 
Ratio  QQ/Q1  T  -  MSj 


Accept-Reject 

Failures 


Reject 
(Equal 
or  More) 

Accept 
(Equal 
or  Less) 

37 

36 

26 

25 

18 

17 

14 

13 

6 

5 

-  6 

5 

4 

3 

3 

2 

★NOTE:  C  refers  to  Revision  C  of  MII-STD-781. 
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How  To  Use  Appendix  B,  Table  6.  As  an  example,  suppose  that  the  upper 
teat  MTBF  is  900  hours  and  the  lower  test  MTBF  is  300,  so  that  the  discrimina¬ 
tion  ratio  (d)  is  900/300  =  3.  Consumer  and  producer  risks  of  approximately 
10%  for  each  are  required.  Now,  as  shown  in  Figure  8-4,  test  plan  XVC  has  a 
and  0  risks  of  9.4%  and  9.9%,  respectively,  and  the  discrimination  ratio  is  3. 
Teat  plan  XVC  requires  a  test  length  (T)  of  9.3  times  the  lower  MTBF  of  300, 
so  T  s  (9. 3) (300)  =  2790  hours.  The  acceptance  criterion  is  to  accept  with  5 
or  fewer  failures  and  reject  with  6  or  more  failures.  Note  that  if  the  upper 
teat  MTBF  had  been  90  and  the  lower  test  MTBF  had  been  30,  the  same  test  plan 
is  appropriate.  However,  in  this  situation  the  test  duration  (T)  is  (9.3) (30) 
or  279  hours,  whereas  the  accept/ reject  criterion  remains  the  same. 

Case  Study  8-4  is  another  example  illustrating  the  use  of  this  table. 

Appendix  B,  Table  7:  Supplemental  Exponential  Test  Plans.  This  table 
presents  information  which  supports  the  development  Of  a  test  plan  based  on 
combinations  of  a  and  0  risks  of  10%,  20%,  and  30%.  Figure  8-5,  in  conjunc¬ 
tion  with  the  following  example  problem,  illustrates  the  use  of  Appendix  B, 
Table  7.  v. 

How  to  Use  Appendix  B,  Table  7.  Concerning  Figure  8-5  and  Appendix  B, 
Table  7,  note  the  following: 

If  the  discrimination  ratio,  d,  is  not  given  exactly  in  the  tables,  going  to 
the  next  lower  value  will  give  a  conservative  (i.e.,  longer)  test  time 
requirement. 

Consider  once  again,  the  example  where  6Q  =  900,  0j  =  300,  and  the  desired  a 

and  0  risks  are  10%  each.  Recall  that  the  discrimination  ratio  was  3.  To 
select  a  test  plan  from  Figure  8-5,  we  search  the  column  labeled  as  a  10% 
producer's  risk  to  find  the  number  closest  to  3.  In  this  case,  test  plan  10-6 
has  a  discrimination  ratio  of  2.94.  The  test  duration  is  (9. 27) (300)  or  2781 
hours  with  5  being  the  maximum  acceptable  number  of  failures.  Note  how  this 
plan  compares  with  test  plan  XVC  which  has  the  same  acceptance  criterion, 
requires  2790  hours  of  test  time,  and  has  a  discrimination  ratio  of  3.  Case 
Study  8-5  further  illustrates  the  use  of  this  table. 

Graphical  Representation  of  Test  Planning  Parameters.  Figure  8-6  graph¬ 
ically  illustrates  the  interaction  between  a  and  0  risks  and  test  length  for 
three  commonly  used  discrimination  ratios.  The  graphs  do  not  provide  complete 
test  planning  information  since  no  acceptance  criterion  is  specified.  These 
curves  are  useful  tools  for  conducting  tradeoff  analyses  between  risk  levels 
and  test  length.  Note  that  some  of  the  specific  test  plans  presented  in 
Appendix  B,  Tables  6  and  7  are  displayed  on  the  curves,  i.e.,  30-7,  10-19, 
20-7,  etc. 

To  illustrate  how  the  graphs  may  be  used,  consider  that  6^  =  100  hours,  dQ  = 
150  hours,  and  a  test  duration  of  2500  hours  is  affordable.  To  enter  the 
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graphs,  use  d  =  Qq/Qj  =  150/100  =  1.5  and  T/Oj  =  2500/100  =  25.  Reading  up 
through  the  three  curves,  ve  fine  he  following  risk  combinations: 


a  = 

0.30 

P  = 

0.10 

a  = 

0.20 

P  = 

0.16 

a  = 

0.10 

P  = 

0.26 

If  one  of  these  combinations  is  tolerable,  the  test  length  is  adequate.  To 
reduce  one  or  both  risks,  the  test  duration  must  be  increased.  Tolerating 
greater  risks  permits  reduction  of  the  test  duration.  Case  Study  8-6  further 
illustrates  the  use  of  these  graphs. 

Figure  8-7  is  a  graphical  portrayal  of  the  interaction  between  test  length  and 
rick  when  a  and  p  risks  are  equal.  Curves  for  each  of  the  three  values  (1.5, 
2.0,  3.0)  of  the  discrimination  ratio  appear  on  e  same  graph.  Case  Study 
8-6  illustrates  the  use  of  Figure  8-7. 

Poisson  Distribution  Equations.  When  a  standard  test  plan  for  a  specific  com¬ 
bination  of  6^,6^,  a,  and  p  is  not  available,  the  test  designer  may  use  the 

Poi3son  equations  to  develop  a  test  plan. 

The  following  notation  is  used  in  the  discussion  of  the  Poisson  equation 
technique . 

T  =  Total  test  exposure 

0  a  True  MTBF 

c  =  Maximum  acceptable  number  of  failures 

0q  =  Upper  test  MTBF 

=  Lower  test  MTBF 
a  =  Producer's  risk 


P  = 

P(ac|0)  = 
P(rej|0)  = 


Consumer's  risk 

Pr  lability  of  accepting  the  system  assuming  the  true  MTBF  is 

0. 

Probability  of  rejecting  the  system  assuming  the  true  MTBF  is 

0. 


The  probability  of  acceptance  is  the  probability  that  no  more  than  a  certain 
(acceptable)  number  of  failures  will  occur.  This  probability  can  be  computed 
using  the  equation: 


P(ac|e) 


c 

1 

k=0 


(T/0)ke“(T/6) 

ki 


(8.7) 
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This  is  the  Poisson  Distribution  Equation.  This  distribution  and  assumptions 
regarding  its  applications  are  discussed  in  Chapter  5. 

The  consumer's  risk  ({3)  is  the  probability  that  during  the  test  no  more  than 
the  acceptable  number  of  failures  will  occur  when  the  true  MTBF  is  6.  .  Conse¬ 
quently,  1 

p  =  PUcjflsBj) 

=  P(c  or  fewer  failures  0=0 j)  , 


where  c  is  the  maximum  acceptable  number  of  failures.  Thus, 

c  (T/0.)k  e'(T/V 

1  - — -  • 

k=0  KI 


(8.8) 


The  producer's  risk  (a)  is  the  probability  that  during  the  test  more  than  the 
acceptable  number  of  failures  will  occur  when  the  true  MTBF  is  0Q.  Conse¬ 
quently, 


a  =  P(rej  0=0Q) 


=  P(c  +  1  or  more  failures  0-0q) 


Since 


P(c  +  1  or  more  failures  0=6q)  =  1  -  P(c  or  fewer  failures  0=0q)  , 


we  have  that 


c  (T/6n)k  e'(T/V 

«=  1  -  *  — Sri - 

k=0  K! 


or  equivalently, 

c  (T/0  )k  e‘(T/V 

1  '  “  =  1  - ^-kl - 

k-0  K! 


(8.9) 


In  order  to  determine  the  complete  test  plan,  we  must  solve  equations  8.8  and 
8.9  simultaneously  for  T  and  c. 


Solving  these  equations  directly  without  the  aid  of  a  computer  is  too  tedious 
and  time  consuming  to  be  considered  practical.  We  therefore  present  the  fol¬ 
lowing  graphical  solution  procedure  vhich  utilizes  the  Poisson  Chart,  Chart 
No.  1  in  Appendix  B. 
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Graphical  Poisson  Solution  Procedure.  We  wish  to  find  a  test  exposure, 
T,  and  an  acceptable  number  of  failures,  c,  such  that  the  probability  of 
acceptance  is  3  when  0  =  and  1  -  a  when  0  =  0q.  This  is  done  graphically 

with  the  use  of  a  transparent  overlay. 

On  an  overlay  sheet,  draw  vertical  lines  at  0/T  =  1  and  0/T  =  Draw 

horizontal  lines  at  probabilities  (3  and  1  -  a,  forming  a  rectangle.  Slide  the 
.overlay  rectangle  horizontally  until  a  curve  for  a  single  value  of  c  passes 
through  the  lower  left  and  upper  right  comers .  (It  may  not  be  possible  to 
hit  the  comers  exactly.  Conservative  values  of  c  will  have  curves  that  pass 
through  the  horizontal  lines  of  the  rectangle.)  This  value  of  c  is  the  ac- 
--ptable  number  of  failures.  Read  the  value  of  0/T  corresponding  to  the  left 
of  the  rectangle.  Divide  0^  by  this  value  to  find  T,  the  required  test 

.xposure.  The  following  numerical  example  illustrates  the  use  of  the  graph¬ 
ical  Poisson  Solution  Procedure. 

We  wish  to  find  the  required  test  exposure,  T,  and  acceptable  number  of  fail¬ 
ures  c;  such  that  when  the  MTBF,  0  =  0^*  100  hours,  the  probability  of  ac¬ 
ceptance,  0,  will  be  0.20  and  then  0  =  0q  =  300  hours  the  probability  of 
acceptance,  1  -  a,  will  be  0.90. 

An  overlay  rectangle  is  constructed  as  shown. 


FIGURE  8-8  OVERLAY  CONSTRUCTION  TECHNIQUE 


a  1.0  3.0 

»T 


Sliding  the  rectangle  to  the  left,  we  find  that  when  c  =  3  the  fit  is  close, 
but  slightly  higher  risks  must  be  tolerated.  Going  to  c  =  4,  the  curve  passes 
through  the  horizontal  lines  of  the  rectangle.  At  the  left  of  the  rectangle, 
0/T  =  0.14,  so  the  required  test  exposure  is  approximately  100/0.14  =  714 
hours  and  the  acceptance  criterion  is  4  or  fewer  failures. 
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FIGURE  8-9  OVERLAY  CURVE  MATCHING  PROCEDURE 


OPERATING  CHARACTERISTIC  (PC')  CURVES 
Introduction 

In  the  previous  sections  of  this  chapter,  we  have  discussed  methods  for  de¬ 
veloping  test  plans  which  achieve  required  a  and  3  risks.  The  test  plan 
itself  is  specified  by  the  test  exposure  and  the  maximum  acceptable  number  of 
failures.  For  a  test  plan  developed  using  the  methods  in  this  chapter,  we 
know  that  the  producer's  risk  (the  probability  of  rejecting  a  good  system)  for 
the  specified  value  (SV)  is  a  and  the  consumer's  risk  (the  probability  of 
acceptance)  for  the  minimum  acceptable  value  (MAV)  is  p.  In  addition,  to 
assess  a  test  plan  proposed  by  another  party,  we  have  shown  methods  for  com¬ 
puting  the  producer's  risk  and  the  consumer's  risk  for  the  SV  and  MAV,  re¬ 
spectively.  A  graphical  tool  which  provides  more  complete  information  about  a 
specific  test  plan  is  the  operating  characteristic  (OC)  curve.  The  OC  curve 
displays  both  acceptance  and  rejection  risks  associated  with  all  possible 
values  of  the  reliability  parameter  and  not  merely  the  SV  and  MAV.  By  def¬ 
inition,  an  OC  curve  is  a  plot  of  the  probability  of  acceptance  (the  ordinate) 
versus  the  reliability  parameter  value  (the  abscissa). 

Figure  8-10  contains  the  operating  OC  curve  for  test  plan  XVICC  from  MIL- 
STD  781C,  with  0^,  the  lower  test  MTBF,  assumed  to  be  100  hours. 

Consider  a  single  point  on  the  curve,  say  an  MTBF  of  200  hours  and  a  prob¬ 
ability  of  acceptance  of  0.63.  This  means  that  for  test  plan  XVIIC  (test 
duration  of  430  hours,  accept  with  2  or  fewer  failures),  a  system  which  has  a 
true  MTBF  of  200  hours  has  a  63%  chance  of  passing  this  test,  i.e.,  being 
accepted.  A  system  requires  an  MTBF  of  around  400  hours  in  order  for  the 
producer  to  be  at  least  90%  confident  that  the  system  will  be  accepted.  A 
system  whose  true  MTBF  is  about  80  hours  has  only  a  10%  chance  of  being  ac¬ 
cepted. 
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FIGURE  8-10  OPERATING  CHARACTERISTIC  (OC )  CURVE 


MTBF,  HOURS 

FOR  THIS  EXAMPLE  0,  IS  ASSUMED  TO  EQUAL  100  HOURS 


Operating  characteristic  curves  for  all  the  test  plans  in  Appendix  B,  Table  6 
of  this  text  can  be  found  in  Appendix  C  of  MIL-STD  781C.  However,  OC  curves 
for  the  test  plans  in  Appendix  B,  Table  7,  of  this  text  are  not  available  in 
MIL-STD  781C. 

OC  Curve  Construction 

The  OC  curve  shown  in  Figure  8-10  is  a  representation  of  the  mathematical 
model  used  to  compute  the  reliability  for  a  system.  We  have  discussed  two 
basic  models  in  previous  sections.  The  Poisson/ exponential  model  is  used  for 
systems  undergoing  continuous  time  testing  and  the  binomial  model  is  used  for 
discrete  time  tests. 

The  OC  curve  specifically  displays  the  relationship  between  the  probability  of 
acceptance  and  MTBF. 


8-18 


ipp|^»»s«*wir7i  WWK  ■  iswr 


For  the  Poisson/ exponential  model,  we  indicated  in  equation  8.7  that 

p(,c|e)  =  ^  — 


(8.10) 


where 


k  =  Number  of  failures 


T  =  Total  test  time 


6  =  Values  of  MTBF 


c  -  Maximum  acceptable  rnunber  of  failures 

Referring  to  Figure  8-10,  let  us  assume  that  6^  =  100  hours  and  that  we  have 

selected  test  plan  XVIIC  from  Appendix  B,  Table  6  which  permits  a  maximum  of 
two  failures.  Using  6^  =  100  hours,  which  corresponds  to  a  test  duration  of 

430  hours  for  test  plan  XVIIC  (T  =  4.30j),  and  c  =  2,  we  can  determine  points 

on  the  curve  by  calculating  P(ac|6)  for  different  values  of  6.  As  an  example, 
for  0=  215  hours 


P(ac  0=215)  = 


/430V  - 
-  \215/  e 
0! 


'430 V  - 
<215/  e 
1! 


a«)V» 


,0-2  ,1-2  ,2-2 

0  12 


=  0.135  +  2(0.135)  +  2(0.135) 


=  0.676  . 

By  choosing  a  sufficient  number  of  values  for  0  between  0  and  500  and  comput¬ 
ing  the  probability  of  acceptance  for  each,  we  can  construct  a  smooth  curve. 

For  the  binomial  model,  the  probability  of  acceptance  is  expressed  by  the 
equation 


P(acjp)  =  I  (J)pkCl-P)1 


c 

=  l 
k=0 


pka-p)n'k 


where 


n  =  Number  of  trials 
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c  =  Maximum  acceptable  number  of  failures 

p  =  Probability  of  failure  on  any  trial. 

By  inserting  specific  values  for  n,  k  and  by  varying  the  probability  of  fail¬ 
ure  on  any  trial,  p,  we  can  compute  values  of  the  probability  of  acceptance 
which  permit  us  to  construct  an  OC  curve. 

For  example,  by  letting  n  =  5  and  c  =  2,  calculate  the  probability  of  accept¬ 
ance  for  p  =  0.1. 

P(ac |p=0. 1)  =  Qf^3~ojr  (0.1)°(1-0.1)5"0  +  jjffqyj-  (0. D^l-O. I)5"1 

+  rutWr  (0*1)2(1"0il)5‘2 

(120) (1) (0 . 9)5  .  (120)(0.1)(0.9)A  .  (120) (0. 1)2(0.9)3 
120  24  (2TT6) 

=  (0 - 9 ) 5  +  (0.5)(0.9)4  +  (0. 1) (0.9)3 
=  0.59  +  0.33  +  0.07  =  0.99. 


Thus, 

P(ac |p=0. 1)  =  0.99  . 

As  expected,  the  probability  of  acceptance  is  very  high  since  we  have  designed 
a  relative  easy  test  to  pass. 
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CASE  STUDY  NO.  8-1 


Background 

A  mechanical  system  which  controls  the  aiming  point  of  a  large-caliber  gun  is 
under  development.  The  specified  and  minimum  acceptable  values  for  the  prob¬ 
ability  of  aiming  correctly  are  0.85  and  0.70  respectively.  Testing  requires 
that  expensive  projectiles  be  fired  for  each  trial,  and  only  20  rounds  are 
allotted  for  testing. 

Determine 


1.  Propose  a  test  plan  which  equalizes  the  consumer's  and  producer's  risks. 
What  are  the  risks? 

2.  The  user  can  tolerate  a  risk  of  no  worse  than  5%.  What  test  plan  gives 
the  best  (smallest)  producer's  risk? 

Solutions 


1.  As  mentioned  in  Chapter  6,  "Statistical  Concepts",  consumer's  risk  in¬ 
creases  when  producer's  risk  decreases,  and  vice  versa,  when  the  sample  size 
is  fixed.  Theoretically,  there  is  a  point  where  they  are  equal  or  almost 
equal . 

It  is  also  important  to  understand  that  the  analytical  interpretation  of  pro¬ 
ducer's  and  consumer's  risk  when  determining  the  solution  to  question  no.  1. 
The  producer's  risk  is  the  probability  of  rejecting  a  system  which  meets  the 
SV  of  0.15  proportion  of  failures  (reliability  of  0.85).  Tor  a  given  accept/ 
reject  criterion  (determined  by  a  value  c  which  represents  the  maximum  number 
of  failures  which  results  in  acceptance  of  the  system),  the  producer's  risk, 
a,  is  the  probability  that  c  +  1  or  more  failures  occur.  The  consumer's  riks 
is  the  probability  of  accepting  a  system  which  exceeds  the  MAV  of  0.30  propor¬ 
tion  of  failures  (reliability  of  0.70).  For  the  same  accept/reject  criterion, 
the  consumer's  risk,  p,  is  the  probability  that  c  or  fewer  failures  occur. 
Below  is  a  section  of  binomial  tables  for  n  -  20,  extracted  from  Appendix  B, 
Table  1 . 


P  = 

1-a  - 

a  = 

P(c  or  fewer  failures) 

P(c  or  fewer  failures) 

P(c+1  or  more  failures) 

c 

P1  =  0.30 

P0  =  0.15 

o 

ii 

o 

1— * 
Ln 

0 

0.00 

0.04 

0.96 

1 

0.00 

0.18 

0.82 

2 

0.03 

0.40 

0.60 

3 

0.11 

0.65 

0.35 

Dl 

0.24 

0.83 

0.17 

5 

0.42 

0.93 

0.07 

6 

0.61 

0.98 

0.02 

7 

0.77 

0.99 

0.01 

The  proposed  test  plan  is  to  accept  with  4  or  fewer  failures  and  reject  with  5 
or  more  failures.  The  consumer's  and  producer's  risks  are  0.24  and  0.17, 
respectively. 

2.  From  the  above  table,  we  see  that  test  plans  which  have  the  maximum 
acceptable  number  of  failures  (c)  of  0,  1,  and  2,  and  satisfy  the  consumer's 
risk  of  no  more  than  5%.  The  best  (smallest)  producer's  risk  occurs  when 
c  =  2,  the  risk  being  0.60. 


CASE  STUDY  NO.  8-2 


Background 

A  new,  highly  reliable  missile  system  is  under  development.  The  specified 
reliability  (SV)  is  0.98,  and  the  minimum  acceptable  reliability  (MAV)  is 
0.85. 


Determine 


1.  Design  test  plans  for  producer's  risks  of  5%,  10%,  and  20%,  with  a  con¬ 
sumer's  risk  of  5% 

2.  Design  test  plans  for  a  producer's  risk  of  10%  and  for  a  consumer's  risk 
of  10%. 

3.  Redo  number  2  if  the  MAV  is  0.92  instead  of  0.85. 

Solutions 


Note  that  a  reliability  of  0.98  corresponds  to  a  proportion  of  failures,  or 

unreliability,  of  0.02,  and  a  reliability  of  0.85  corresponds  to  a  proportion 

of  failures,  or  unreliability,  of  0.15.  Thus,  we  list  our  test  planning 
parameters  pQ  and  Pj  as  0.02  and  0.15,  respectively. 

l.a.  P0=0.02  p1  =  0.15  of  =  0.05  p  «  0.05 

i.  Normal  Approximation.  In  order  to  determine  a  starting  point  for 

our  analysis,  we  calculate  approximate  values  of  n  and  c  using 

equations  8.3  and  8.4.  For  values  of  z  and  zft,  use  Appendix  B, 

Table  2.  a  p 

n  =  {(1.645)2(0. 02-0. 0004)  +  (1.645)2(0. 15-0.0225) 

+  2(1. 645)2  V(0.02) (0 . 15) (0. 98) (0.85) }/(0. 15-0.02)2 
=  39.6 

c  =  (1.645)  V(d9 >6) (0.02) (0.98)  +  (39.6)(0.02)  -  0.5 


=  1.7 


8-23 


ii.  Poisson  Approximation  (Appendix  B,  Chart  1,  with  np  =  T/6,  the 
reciprocal  of  8/T) 


apl  n  ^0  £  a 


0  3.0  20.0  0.40  0.05  0.69  0.32 

1  4.7  34.3  0.63  0.06  0.88  0.12 


3  7.8  52.0  1.04  0.05  0.97  0.03 


NOTE:  a  =  P(c  +  1  or  more  failures) 

0  =  P(c  or  fewer  failures) 

1-a  *  P(c  or  fewer  failures) 

iii.  Proposed  Test  Plans.  It  appears  from  i  and  ii  above  that  a  good 
starting  point  for  fine  tuning  is  an  n  of  40  and  c  of  2.  Using 
Appendix  B,  Table  1  to  fine  tune,  we  propose  the  following  test 
plans . 


n 

c 

a 

§ 

40 

2 

0.04 

0.05 

39 

2 

0.04 

0.05 

38 

2 

0.04 

0.06 

37 

2 

0.04 

0.07 

The  protection  afforded  by  this  plan  seems  to  be  adequate  though  the 
consumer's  risk  is  8%  (slightly  above  the  required  5%). 

l.b.  pQ  =  0.02  px  =  0.15  a  =  0.10  p  =  0.05 
i.  Normal  Approximation 


ii.  Poisson  Approximation  (Appendix  8,  Chart  1) 


1 » c* 


c 

npl 

n 

§ 

l-q  q 

0 

3.0 

20.0 

0.40 

0.05 

0.69  0.31 

1 

4.7 

34.3 

0.63 

0.05 

0.88  0.12 

2 

6.3 

42.0 

0.84 

0.05 

0.94  0.06 

3 

7*8 

52.0 

1.04 

0.05 

0.97  0.03 

iii.  It  appears 

that 

a  good  starting  point  for  fine  tuning  is  an  n 

and  c  of  1. 

The  following  test  plans  are  proposed. 

n 

c 

a 

§ 

35 

1 

0.15 

0.03 

34 

1 

0.15 

0.03 

33 

1 

0.14 

0.03 

32 

1 

0.13 

0.04 

31 

1 

0.13 

0.04 

30 

1 

0.12 

0.05 

29 

1 

0.11 

0.05 

*[ 

28 

1 

0.11 

0.06 

* 

The  actual 

risks 

exceed  the 

required  risks  of 

10%  and  5%  but  i 

any  significant  extent. 

pQ  =  0.02 

pl  = 

0.15  a  = 

0.20 

P  =  0.05 

i.  Normal  Approximation 

n  =  {(0.84)J 

•(0.02-0.0004)  +  (1.645)2(0. 15-0.0225) 

+  2(0.84)(1.645)  V(0.02)(0. 15) (0.98)70. 85) }/(0. 15-0.0: 

=  29.4 

o 

o 

Vw/ 

<r 

o* 

'w' 

>4* 

CO 

o 

II 

u 

)(0.98) 

'  +  (29.4) (0.02)  -  0.5 

-  0. 

72 

ii.  Poisson  Approximation  (Appendix  B, 

Chart  1) 

c 

npj 

n 

£ 

l-q  q 

0 

3.0 

2.0 

0.40 

0.05 

0.69  0.31 

1 

4.7 

34.3 

0.63 

0.05 

0.88  0.12 
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iii.  It  appears  that  a  good  starting  point  for  fine  tuning  is  an  n  of  25 
and  c  of  1.  The  following  programs  are  proposed. 


*  |_25 _ 1_ _ 0.09  0.09 

25  0  0.40  0.02 

^Generally,  there  is  no  reasonable  test  plan  for  the  input  values 
given.  A  very  large  sample  size  is  required  to  achieve  an  a  of 
0.20.  (For  n  =  40,  a  =  0.19,  and  P  =  0.01,  with  a  c  of  0.)  This 
sample  size  seems  unwarranted.  Our  recommendation  is  to  use  the  n 
of  25  and  the  c  of  1. 

pQ  =  0.02  p  =0.15  a  -  0.10  p  =  0.10 

i.  Normal  Approximation 

n  =  {(1.28)2(0. 02-0. 0004)  +  (1 .28)2(0. 15-0.0225) 

+  2(1. 28)2  V (0 - 02 )  ( 0 .15 )  (0 . 98 )'( 0785) } / ( 0 . 15 -0 . 02 ) 2 
=  24.2 

c  =  (1.28)  V(24. 25 (0.02) (0.98)  +  (24.2)(0.02)  -  0.5 

=  0.86 

ii.  Poisson  Approximation  (Appendix  B,  Chart  1) 


c 

npj 

n 

^0 

£ 

1-q 

0 

2.3 

15.3 

0.31 

0.10 

0.74 

1 

3.9 

26.0 

0.52 

0.10 

0.90 

2 

5.2 

35.0 

0.70 

0.10 

0.96 

iii.  It  appears  that  a  good  starting  point  for  fine  tuning  is  au  n  of  25 
and  c  of  1.  The  following  programs  are  proposed. 


*  [jib _ 1 _ 0.09 _ 0.09 

24  1  0.08  0.11 

Sr 

The  test  plans  with  a  sample  size  of  25  fits  well.  The  sample  size 
can  be  reduced  by  1  to  24  if  the  consumer  allows  his  risk  to  be  11%. 


.  ' . ... 


p0  =  0.02  p1  =  0.08  a  =  0.10  p  =  0.10 


i.  Normal  Approximation 

n  =  {(1.28)2(0. 02-0. 0004)  +  (1 .28)2(0. 08-0. 0064) 

+  2C1.28)2  i/(0.Q2) (0.08) (0.98) (0.92) }/ (0.08-0. 02) 2 
=  77.0 

c  =  (1.28)  V (77 . 0) (0 .02) (0 .98)  +  (77.0)(0.02)  -  0.5 
-  2.6 

ii.  Poisson  , proximation  (Appendix  B,  Chart  1) 


c 

np. 

_ 

nPo 

ft. 

1-q 

u 

1 

"  .9 

48.8 

0.87 

0.10 

0.75 

0.25 

2 

5.3 

66.3 

1.32 

0.10 

0.88 

0.12 

1  3 

6.7 

83.8 

1.67 

0.10 

0.91 

0.09  ] 

4 

8  0 

100.0 

2.00 

0.10 

0.94 

0.06 

iii.  It  appears  that  a  good  starting  point  for  fine  tunirg  is  an  n  of  75 
and  c  of  3.  The  following  programs  are  proposed. 


n 

c 

a 

ft 

75 

3 

0.06 

0.14 

76 

3 

0.07 

0.13 

77 

3 

0.07 

0.13 

78 

3 

0.07 

0.13 

79 

3 

0.07^ 

0.11 

80 

3 

0.08 

0.10 

♦ 
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CASE  STUDY  NO.  8-3 


Background 

An  operational  test  is  being  considered  for  a  disposable  survival  ratio  which 
must  work  for  at  least  three  hours.  The  ratio  has  a  specified  mission  reli¬ 
ability  of  0.85  and  a  minimum  acceptable  mission  reliability  of  0.7.  A  number 
of  radios  will  be  put  on  test  for  three  hours  each  and  the  number  of  failures 
recorded . 

Determine 


1.  Propose  some  test  plans  for  a  producer's  risk  of  about  10%  and  consumer's 
risks  of  about  10%,  20%,  and  30%. 

2.  Propose  a  test  plan  for  a  minimum  acceptable  reliability  of  0.5  with  a 
user  risk  of  2%  and  a  producer  risk  of  20%. 

Solutions 


Note  that  a  reliability  of  0.85  dorresponds  to  a  proportion  of  failures  or 
"unreliability"  p^  of  0.15. 

l.a.  p0  =  0.15  pj  =0.3  a  =  0.10  0  =  0.10 

i.  Normal  Approximation 

n  =  {(1.28)2(0. 15-0  0225)  +  (1.28)2(0.3-0.09) 

+  2(1. 28)2  V(0* 15) (0 .3) (0.85) (0. 7) }/(0.3-0 . 15)2 
=  48.4 

c  =  1.28  V (48 >4) (O'  15) (0.85)  +  (48.4)(0.15)  -  0.5 


=  9.9 


ii.  Poisson  Approximation  (Appendix  B,  Chart  1) 


£ 

aPl 

n 

npg 

& 

1-ot 

a 

7 

12.0 

40.0 

6.0 

0.10 

0.75 

0.25 

8 

13.0 

43.3 

6.5 

0.10 

0.80 

0.20 

9 

14.0 

46.7 

7.0 

0.10 

0.83 

0.17 

l1G 

15.5 

51.7 

7.7 

0.10 

0.85 

0.15 

11 

16.5 

55.0 

8.2 

0.10 

0.88 

0.12 

12 

18.0 

60.0 

9.0 

0.10 

0.89 

0.11 

13 

19.0 

63.0 

9.5 

0.10 

0.90 

0.10 
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iii.  It  appears  that  a  good  starting  point  for  fine  tuning  is  an  n  of  50 
and  c  of  10.  The  following  programs  are  proposed. 


pQ  =  0.15  Pj  =0.3  a  =  0.10  0  =  0.20 

i.  Normal  Approximation 

n  =  {(1.28)2(0. 15-0.0225)  +  (0.84)2(0.3-0.09) 

+  2(0.84) (1 .28)  V(0.15)(0.3)(0.85)(0.7)}/(0.3-0.15): 
=  31.5 

c  =  (1.28)  /(31.5)(0.15)(0.85)  +  (31 . 3) CO . 15)  -  0.5 
=  6.78 

ii.  Poisson  Approximation  (Appendix  B,  Chart  1) 


’.ii.  It  appears  that  a  good  starting  point  for  fine  tuning  is  an  n  of  35 
and  c  of  7.  The  following  programs  are  proposed. 


n 

c 

Of 

§ 

35 

7 

0.14 

0.13 

34 

7 

0.12 

0.16 

33 

7 

0.11 

0.19 

32 

7 

0.10 

0.21 

c  npl  n  np0  g  • 1-g  a 

4  5.9  19.7  2.9  0.30  0.82  0.18 

6  8.4  28.0  4.2  0.30  0.89  0.11 

iii.  It  appears  that  a  good  starting  point  £or  fine  tuning  is  an  n  of  22 
and  c  of  5.  The  following  programs  are  proposed. 

n  c  n  g 


2. 

i. 


23  5  0.12  0.27 

Pq=0.15  p.  =0.5  a  =  0.20  p  =  0.02 

Normal  Approximation 

n  =  {(0.84)2(0. 15-0.0225)  +  (2.06)2(0.5-0.25) 

+  2(0.84)(2.06)  ^(0 . 15) (0.5) (0. 85) (0 .5) }/ (0.5-0. 15) 2 


=  14.4 


ii.  Poisson  Approximation  (Appendix. 8,  Chart  6) 


c 

npj 

n 

npQ 

§ 

1-a 

a 

2 

7.5 

15.0 

2.25 

0.02 

0.63 

0.37 

3 

9.1 

18.2 

2.70 

0.02 

0.73 

0.27 

4 

10.6 

21.2 

3.20 

0.02 

0.80 

0.20 

iii.  It  appears  that  a  good  starting  point  for  fine  tuning  is  an  n  of  15 
and  c  of  3.  The  following  programs  are  proposed. 

n  £  a  £ 

15 _ 3 _ 0JL8 _  0 .02 

14  3  0.16  0.03 


% 
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CASE  STUDY  NO.  8-4 


Background 

A  communication  system  has  minimum  acceptable  value  (MAV)  of  100  hrs  MTBF,  and 
a  specified  value  (SV)  of  150  hrs  MTBF. 

Determine 


How  many  hours  of  test  are  required  for  design  qualification  prior  to  a  pro¬ 
duction  decision  if  we  desire  a  and  f)  risks  of  10%  each? 

Solution 

0O  =  150,  ex  =  100,  d  =  y§§  =  1.5,  0.10 

In  this  case,  a  "standard"  test  plan  may  be  selected  from  Appendix  B,  Table  6. 
Test  plan  IXC  satisfies  these  inputs.  The  required  test  duration  is 

T  =  (45.0) (01)  =  (45.0) (100)  =  4500  hrs. 

The  accept/reject  criterion  for  this  test  plan  is  to  accept  if  we  encounter  36 
or  fewer  failures.  Bear  in  mind  though,  that  the  acceptance  criteria  may 
require  violation,  depending  on  the  nature  of  the  failures  and  the  verifica¬ 
tion  of  corrective  action. 

Commentary 

1.  In  order  to  make  the  test  less  sensitive  to  the  constant  failure  rate 
assumption,  it  would  be  desirable  to  have  at  least  3  systems  tested  for  at 
least  3(100)  =  300  hours  each.  The  remainder  of  the  4500  hours  may  be  satis¬ 
fied  with  these  or  other  systems.  See  section  entitled  "Constant  Failure  Rate 
Assumption"  for  a  discussion  of  this  topic. 

2.  The  test  duration  of  4500  hours  is  very  long!  (The  equivalent  of  187.5 
24-hour  days).  Putting  more  systems  on  test  will  reduce  the  calendar  time 
requirement,  but  4500  hours  of  test  exposure  jre  still  required.  The  required 
test  exposure  is  high  because  of  the  low  discrimination  ratio,  d,  and  the 
relatively  low  a  and  0  risks.  Plans  with  higher  risks  may  be  worth  considera¬ 
tion  to  ree  how  much  the  amount  of  testing  may  be  reduced. 
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CASE  STUDY  NO.  8-5 


Background 

An  air  defense  system  has  a  minimum  acceptable  value  (MAV)  of  80  hours  MTBF 
and  a  specified  value  (SV)  of  220. 

Determine 


How  many  hours  of  testing  are  required  to  give  the  user  80%  assurance  that  his 
needs  have  been  met?  The  producer  requires  <90%  assurance  that  his  product 
will  be  accepted  if  it  meets  the  SV. 

Solution 


The  80%  user  assurance  is  equivalent  to  a  consumer's  risk  of  (3  =  0.20,  and  the 
90%  contractor  assurance  is  equivalent  to  a  producer's  risk  of  a  =  0.10. 

0O  =  220,  6:  =  80,  d  =  ^|~  =  2.75. 

Because  the  discrimination  ratio  is  2.75,  the  "standard"  test  plans  from 
Appendix  B,  Table  6,  cannot  be  used.  Appendix  B,  Table  7,  will  be  considered. 

For  the  20%  0  risk,  and  entering  the  10%  Of  risk  column,  we  find  a  discrimina¬ 
tion  ratio  of  2.76  available,  which  is  very  close  to  2.75.  This  is  test  plan 
number  20-5.  The  required  test  duration  is 

T  =  (6.72)(61)  =  (6.72)(80)  =  537.6  hrs. 

The  accept/ reject  criterion  is  to  accept  with  4  or  fewer  failures. 
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CASE  STUDY  NO.  8-6 


Background 

A  radar  system  is  under  development.  A  test  is  to  be  run  which  will  be  es¬ 
sentially  a  fixed  configuration  test.  At  this  stage  of  development,  an  MTBF 
of  200  hours  is  planned,  but  assurance  is  desired  that  the  MTBF  is  not  lower 
than  100  hours .  For  cost  and  schedule  reasons ,  a  test  exposure  of  300  hours 
has  been  proposed. 

Determine 

Is  this  amount  of  test  exposure  adequate?  If  not,  what  is  an  adequate  amount 
of  testing? 

Solution 


The  upper  test  value,  0Q,  is  200,  and  the  lower  test  value,  6.,  is  100.  Test 
exposure,  T,  is  300  hrs. 

For  a  quick  and  simple  look  at  the  adequacy  of  the  proposed  test,  we  will  use 
Figure  8-7.  (For  the  convenience  of  the  reader,  the  graphs  in  Figures  8-6  and 
8-7  have  been  reproduced  and  annotated  below.)  Entering  Figure  8.7  with  T/Oj 

=  300/100  =  3  and  d  -  6q/8|  s  200/100  =  2,  we  find  that  the  proposed  test 

exposure  results  in  risks  slightly  above  30%.  The  amount  of  testing  proposed 
is  minimally  adequate  for  this  stage  in  the  program. 

We  may  use  Figure  8-7  to  determine  an  adequate  test  duration.  At  about  580 
hours,  of  =  0  =  0.25.  At  about  900  hours,  a  -  0  s  0.20.  At  about  2000  hours, 
a  =  P  s  0.10.  At  580  hours,  the  risks  are  fairly  high.  A  0  risk  of  25%  is 
perhaps  tolerable,  but  an  Of  risk  of  25%.  means  a  25%  chance  of  an  erroneous 
"back  to  the  drawing  board"  decision. 

A  test  duration  of  about  900  hours  looks  reasonable,  particularly  if  we  reduce 
a  by  letting  0  increase.  To  investigate  this  possibility,  we  may  use 
Figure  8-6.  From  the  graph  for  d  =  2,  we  find  that  test  plan  30-8  with 
of  -  0.10,  0  =  0.30  and  T  =  981  looks  reasonable.  (Test  plan  30-5  with 
a  -  0.20,  0  =  0.30  and  T  =  589  is  another  attractive  possibility).  A  test 
duratiou  of  about  900  hours  is  recommended. 

Commentary" 

The  process  of  trading  test  length  for  testing  risks  is  inherently  somewhat 
subjective.  Actual  problems  of  the  type  illustrated  in  this  case  should,  of 
course,  explicity  address  time,  cost,  and  other  penalties  associated  with  the 
test. 
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CASE  STUDY  NO.  8-7 


Background 

A  program  manager  desires  to  demonstrate  an  MTBF  of  at  least  200  at  a  90% 
confidence  level. 

Determine 

The  minimum  test  length  required,  and  an  evaluation  of  the  proposed  test  plan. 
Commentary 

The  correct  interpretation  of  the  background  statement  is  that  the  MAV  is  200 
hours  and  the  consumer's  risk  is  10%. 

Solution 

The  absolute  minimum  test  length  is  that  which  permits  making  the  desired 
confidence  statement  with  zero  failures.  Applying  inequality  7.10a  we  have: 


•23 


xa,2r+2 


-  2T 
"x2 
*0.10,2 


of  test  exposure. 


=  460.0  hours 


NOTE:  Values  of  2r+2  are  *oun<*  in  Appendix  B,  Table  5. 

An  MTBF  of  200  can  be  demonstrated  at  a  90%  confidence  level  by  completing 
460.0  hours  of  test  exposure  with  zero  failures. 

To  evaluate  this  proposed  test  plan,  we  will  use  an  OC  curve.  To  construct 
the  0C  curve,  we  use  equation  8.10. 

P(ac|6)  -  Probability  of  acceptance  for  a  given  value  of  6 

P(.c|6)  =  l  tT/>)  . 

k-0  k! 
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Commentary 

The  curve  shows  that  the  proposed  test  plan  does,  in  fact,  achieve  a  con¬ 
sumer's  risk  of  10%  for  the  MAV  of  200  hours.  Let  us  now  examine  the  0C  curve 
for  this  test  plan  through  the  eyes  of  the  contractor. 

The  curve  shows  that  a  system  whose  true  MTBF  is  650  hours  has  only  a  50% 
chance  of  passing  the  test,  i.e.,  being  accepted.  In  addition,  for  a  system 
to  have  a  90%  chance  of  being  accepted,  it  must  have  a  true  MTBF  of  4,400 
hours.  In  other  words,  for  the  contractor  to  obtain  a  producer’s  risk  of  10%, 
he  needs  to  manufacture  a  system  whose  true  MTBF  is  4,400  hours.  To  have  a 
50/50  chance  of  passing  the  test,  he  needs  to  manufacture  a  system  whose  true 
MTBF  is  650  hours.  The  lesson  to  be  learned  here  is  that  consideration  must 
be  given  to  both  the  upper  test  value  (SV),  the  lower  test  value  (MAV),  and 
the  risks  associated  with  them  in  designing  or  evaluating  a  test  plan.  The 
test  planner  or  evaluator  should  be  concerned  with  obtaining  the  minimal  test 
exposure  plan  which  protects  both  the  consumer  and  producer.  To  ignore  either 
aspect  can  be  a  dangerous  policy. 
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CHAPTER  9 


ill 


RELIABILITY  GROWTH 


INTRODUCTION 

Initial  prototype  models  of  complex  weapon  systems  will  invariably  have  inher¬ 
ent  reliability  and  performance  deficiencies  that  generally  could  not  have 
been  foreseen  and  eliminated  in  early  design  stages.  To  uncover  and  eliminate 
these  deficiencies,  we  subject  these  early  prototypes  and  later  more  mature 
models  to  a  series  of  development  and  operational  tests.  These  tests  have 
been  specifically  planned  to  stress  the  system  components  to  predetermined 
realistic  levels  at  which  inadequate  design  features  will  surface  as  system 
failures.  These  failures  are  analyzed,  design  modifications  incorporated,  and 
then  the  modified  system  is  tested  to  verify  the  validity  of  the  design 
change . 

This  testing  philosophy  utilizes  the  tcst-analyze-f ix-test  (TAFT)  procedure  as 
the  basic  catalyst  in  achieving  system  reliability  growth.  The  ultimate  goal 
of  a  reliability  growth  program,  and,  ideed,  the  entire  test  program,  is  to 
increase  system  reliability  to  stated  requirement  levels  by  eliminating  a 
sufficient  number  of  inherent  system  failure  modes. 

A  successful  system  reliability  growth  program  is  dependent  on  several 
factors.  First,  an  accurate  determination  must  be  made  of  the  current  system 
reliability  status.  Second,  a  test  program  must  be  planned  which  subjects  the 
system  to  test  exposure  and  stress  levels  adequate  to  uncover  inherent  failure 
models  and  to  verify  design  modifications.  Third,  the  program  manager  must 
address  the  availability  of  test  schedule  and  resource  required  to  support  the 
"TAFT"  procedure . 


To  adequately  control  the  above  and  other  factors  inherent  in  the  reliability 
growth  process,  it.  is  important  to  track  reliability  growth  throughout  the 
testing  program.  This  is  accomplished  by  periodically  assessing  system  reli¬ 
ability  (e.g.,  at  the  end  of  every  test  phase)  and  comparing  the  current  reli¬ 
ability  to  the  planned  level  of  achievement  for  that  point  in  time.  These 
assessments  provide  the  necessary  data  and  visibility  to  support  necessary 
corrective  management  initiatives. 

The  following  paragraphs  present  the  analytical  tools  required  to  plan  a 
reliability  growth  program  and  those  useful  in  tracking  the  actual  growth  of  a 
system  during  consecutive  test  phases. 


RELIABILITY  GROWTH  CONCEPTS 
Idealized  Growth 

For  a  system  under  development,  reliability  generally  increases  rapidly  early 
on  and  at  a  much  slower  rate  towards  the  end  of  development.  It  is  useful  at 
the  beginning  of  a  development  program  to  depict  the  growth  in  reliability  as 
a  smooth  curve  which  rises  at  slower  and  slower  rates  as  time  progresses. 
This  curve,  known  as  the  idealized  growth  curve,  does  not  necessarily  convey 
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precisely  how  the  reliability  will  actually  grow  during  development.  Its 
purpose  is  to  present  a  preliminary  view  as  to  how  a  program  should  be  pro¬ 
gressing  in  order  for  the  final  reliability  requirements  to  be  realized.  The 
model  for  the  idealized  curve  is  the  Duane  Growth  Model ,  the  primary  feature 
of  which  is  the  every  decreasing  rate  of  growth  as  testing  progresses. 

The  development  testing  program  will  usually  consist  of  several  major  test 
phases.  Within  each  test  phase,  the  testing  may  be  conducted  according  to  a 
program  which  incorporates  fixes  or  design  changes  while  testing  is  in  pro¬ 
cess,  at  the  end  of  the  test  phase,  or  both.  I'i  we  divide  the  development 
testing  program  into  its  major  phases  and  join  by  a  smooth  curve  the  proposed 
reliability  values  for  the  system  at  the  end  of  these  test  phases,  the  re¬ 
sulting  curve  represents  the  overall  pattern  for  reliability  growth.  This  is 
called  the  idealized  reliability  growth  curve.  The  idealized  curve  is  very 
useful  in  quantifying  the  overall  development  effort  and  serves  as  a  sig¬ 
nificant  tool  in  the  planning  of  reliability  growth. 

Planned  Growth 

The  planning  of  reliability  growth  is  accomplished  early  in  the  development 
program,  before  hard  reliability  data  are  obtained,  and  is  typically  a  joint 
effort  between  the  program  manager  and  the  contractor.  Its  purpose  is  to  give 
a  realistic  and  detailed  indication  of  how  system  reliability  enhancement  is 
planned  to  [.row  during  development.  Reliability  growth  planning  addresses 
program  schedules,  testing  resources  and  the  test  exposure  levels.  The  ob¬ 
jective  of  growth  planning  is  to  determine  the  number  and  length  of  distinct 
test  phases,  whether  design  modifications  will  be  incorporated  during  ‘or 
between  distinct  test  phases  and  the  increases  in  reliability  to  ensure  that 
the  achieved  reliability  remains  within  sight  of  the  idealized  growth  values. 

Growth  Tracking 

The  primary  objective  in  tracking  reliability  growth  is  to  obtain  demonstrated 
reliability  values  at  the  end  of  each  test  phase.  The  demonstrated  reli¬ 
ability  is  usually  determined  by  one  of  two  methods.  The  first  and  preferred 
method  is  reliability  growth  analysis.  However,  should  the  data  not  lend 
themselves  to  this  type  of  analysis,  then  the  second  method,  an  engineering 
analysis,  should  be  used.  Reliability  growth  analysis  is  useful  for  combining 
test  data  to  obtain  a  demonstrated  estimate  in  the  presence  of  changing  con¬ 
figurations  within  a  given  test  phase.  Engineering  analysis  is  employed  when 
the  reliability  growth  analysis  procedure  is  inappropriate.  We  do  not  address 
engineering  analysis  in  this  text. 


The  first  step  in  planning  reliability  growth  is  the  development  of  an 
idealized  growth  curve.  The  development  of  this  curve  is  based  on  the  fol¬ 
lowing  three  parameters: 

tj  =  length  of  initial  test  phase, 

Mj  =  average  MTBF  over  the  first  test  phase,  tj. 

a  =  a  parameter  which  addresses  the  rate  of  growth. 


The  idealized  curv.» ,  illustrated  in  Figure  9-1,  is  a  graph  of  the  function 
M(t)  where: 

{Mj  in  the  interval  0<t<t^ 

and 

Mi(t/t1)a(l-a)  *  in  the  interval  t>tj  .  (9.1) 


FIGURE  9*1  IDEALIZED  GROWTH  CURVE 


The  idealized  growth  curve  development  procedure  starts  with  the  determination 
of  the  initial  test  phase  length  (tj)  and  the  average  MTBF  over  the  initial 

test  phase  (Mj).  There  is  no  exact  procedure  for  determining  values  of  these 

parameters.  The  initial  test  phase  length  (tj)  may  be  determined  through  a 

joint  effort  of  both  the  contractor  and  the  program  manager.  Perhaps  an 
initial  test  has  already  been  performed,  in  which  case  both  tj  and  Mj  are 

known.  If  this  is  not  the  case,  then  the  determination  of  a  value  for  Mj 

would  in  all  likelihood  require  the  expertise  of  individuals  familiar  with 
present  day  capabilities  of  the  actual  system  in  question  or  other  similar 
systems.  The  parameter,  Mj,  should  be  a  realistic  estimate  of  what  the  sys¬ 
tem's  average  MTBF  will  be  during  the  initial  test  phase,  i.e.,  before  any 
significant  design  weaknesses  can  be  detected  and  modifications  developed, 
implemented  and  tested. 

The  parameter  a  represents  the  rate  of  growth  necessary  to  achieve  an  MTBF  of 
Mg.  (the  contractually  specified  value)  after  a  total  of  T  hours  of  testing. 

The  specified  value  Mj,  represents  the  user's  desired  capability  and  is  deter¬ 
mined  by  means  of  extensive  battlefield  as  well  as  logistics  analyses.  The 


total  amount  of  testing  T  is  a  value  which  is  determined  through  a  joint  con¬ 
tractor  and  program  manager  effort  and  is  based  upon  considerations  of  cal¬ 
endar  time  and  number  of  prototypes  available  in  addition  to  cost  constraints. 
For  fixed  values  of  t^,  Mj.  T,  and  Mj.,  the  value  for  a  is  calculated  alge¬ 
braically  ky  solving  the  equation 

Mj.  =  MI(T/t1)°'(l-a)"1  .  (9.2) 

There  is  no  closed  form  solution  for  a  in  equation  9.2.  However,  an  approxi¬ 
mation  for  a  is  given  below. 

a  =  logfiCtj/T)-!  +  {(loggCT/tp+l)2  +  ilog^Mp/Mj)}172  .  (9.3) 

This  is  a  reasonably  good  approximation  when  a  is  smaller  than  0.4.  The 
approximation  will  always  be  on  the  high  side  but  within  two  decimal  places 
for  values  of  a  less  than  0.3.  Programs  which  require  a  growth  rate  (a) 
greater  than  0.3  should  be  viewed  somewhat  skeptically  and  those  which  require 
an  a  greater  than  0.4  are  far  too  ambitious  to  be  realistic. 

PLANNED  GROWTH  CURVE  DEVELOPMENT 

Once  the  idealized  curve  has  been  constructed,  it  is  used  as  a  basis  for 
developing  a  planned  growth  curve.  The  planned  growth  curve  displays,  in 
graphic  terms,  how  the  producer  plans  by  stages  to  achieve  the  required  final 
MTBF.  The  curve  is  divided  into  portions  which  represent  the  different  test 
phases.  The  entire  curve  indicates  graphically  where  in  the  development 
program  reliability  is  expected  to  grow,  and  where  it  is  expected  to  remain 
constant.  The  curve  depicts  increase?  in  reliability  resulting  from  design 
improvements.  At  any  given  time  during  development  testing,  the  planned 
growth  curve  value  can  be  higher  than,  lower  than,  or  equal  to  the  idealized 
growth  curve  value.  The  idealized  curve  serves  as  a  guide  for  the  preparation 
of  the  planned  curve.  At  no  time  in  the  planning  of  reliability  growth  should 
the  separation  between  values  on  the  curve  be  large.  It  this  is  the  case, 
then  unquestionably  there  is  some  point  during  development  where  an  un¬ 
realistic  jump  in  reliability  is  expected  to  occur. 

As  we  mentioned  earlier,  the  planned  growth  curve  should  graphically  display 
how  reliability  is  expected  to  grow.  Growth,  of  course,  will  generally  occur 
as  a  result  of  incorporating  design  modifications.  These  modifications  may  be 
incorporated  during  the  test  phase,  resulting  in  a  smooth  gradual  improvement 
in  reliability,  or  at  the  end  of  the  test  phase,  resulting  in  a  jump  in  reli¬ 
ability  from  the  end  of  one  test  phase  to  the  beginning  of  the  subsequent  test 
phase.  In  Figure  9-2,  we  present  a  planned  growth  curve  which  illustrates  the 
effect  on  reliability  of  design  improvements  incorporated  during,  and  at  the 
completion  of,  the  various  test  phases.  Note  that  the  rate  of  growth  is 
gradually  decreasing  as  the  system  matures. 

The  portion  of  the  planned  growth  curve  between  time  zero  and  is  identical 
to  the  idealized  growth  curve. 


9-4 


FIGURE  9-2 


PLANNED  GROWTH  CURVE 


Delayed  fixes  are  incorporated  after  each  of  the  first  three  test  phases. 
During  all  of  test  phase  2  and  early  in  test  phase  3,  fixes  are  incorporated. 
Fixes  are  incorporated  during  the  final  test  phase,  and  the  MTBF  grows  to  the 
required  specified  value.  It  is  not  a  good  practice  to  allow  for  a  jump  in 
reliability  at  the  end  of  the  final  test  phase  even  though  fixes  may  be  in¬ 
corporated.  The  reason  is  that  there  is  no  test  time  available  to  determine 
the  impact  of  these  fixes. 

The  planned  growth  curve  is  an  indication  of  how  the  required  MTBF  might  be 
achieved  and  is  developed  by  using  the  idealized  curve  as  a  guide. 

Figure  9-3  illustrates  the  graphical  relationship  between  the  planned  growth 
curve  and  the  corresponding  idealized  curve.  A  point  on  the  planned  curve  at 
any  given  time  in  the  program  represents  the  level  of  reliability  to  be 
achieved  at  that  time. 
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FIGURE  9*3  EXAMPLE  OF  A  PLANNED  GROWTH  CURVE  AND 
CORRESPONDING  IDEALIZED  CURVE 


PHASE  I  t,  PHASE  2  PHASE  3  PHASE  4  T 

TIME 


RELIABILITY  GROWTH  TRACKING 

The  objectives  of  growth  tracking  include: 

-  Determining  if  growth  is  occurring  and  to  what  degree, 

-  Estimating  the  present  reliability,  and 

-  Formulating  a  projection  of  the  reliability  expected  at  some  future 
time. 

The  methods  discussed  in  this  section  are  directed  toward  reliability  growth 
tracking  using  a  mathematical  model.  Parameters  of  the  model  are  estimated 
using  data  which  have  been  accumulated  during  a  given  test  phase.  Using  this 
model  and  the  parameter  estimates,  we  can  determine  present  and  projected 
reliability  values.  The  present  value  represents  the  reliability  inherent  in 
the  existing  configuration.  A  projected  value  represents  the  reliability  of 
the  system  expected  at  some  future  time.  Projected  values  take  into  account 
the  effect  of  design  improvements  intended  to  correct  observed  failure  modes 
or  failure  modes  which  further  testing  will  surface.  Generally  growth  track¬ 
ing  analysis  is  performed  at  the  end  of  a  major  test  phase. 


The  mathematical  model  we  shall  use  for  growth  tracking  describes  the  failure 
rate  as  a  function  of  time.  The  value  r(t)  denotes  the  failure  rate  of  the 
system  after  t  units  of  testing,  and 

r(t)  =  A0tP'\  (9.4) 

where  A  and  0  are  parameters  of  the  model  which  determine  the  scale  and  the 
shape  of  the  curve.  The  reciprocal  of  r(t)  is  the  MTBF  of  the  system  after  t 
units  of  testing.  We  fit  the  model  to  the  actual  test  data  using  maximum 
likelihood  estimates  for  A  and  0.  (See  Chapter  6  for  a  discussion  of  maximum 
likelihood  estimates.) 

When  actual  failure  times  (tj,  t2>  ...  t^)  are  known  and  the  test  phase  is 
time  truncated,  i.e.,  at  time  T,  the  estimate  for  0  is 


0  = 


.  . N _ 

N 

Nlog  T-  1  log  t 
i=l 


(9.5) 


The  estimate  for  A  is 

/N 

A  =  N/TP.  (9.6) 

When  the  test  phase  is  failure  truncated,  i.e.,  at  time  tjj,  the  estimates  are 


and 


_ N _ 

N-l 

(N-l)logetN-  I  logeti 


> 


A  =  N/T^. 

In  either  case,  the  estimate  of  r(t)  is 


r(t)  =  A0t 


0-1 


(9.7) 


(9.8) 


(9.9) 


The  reciprocal  of  r(t)  is  the  estimate  of  the  MTBF  of  the  system  after  a  test 
period  of  length  t,  that  is 

M(t)  =  fit!  * 

Confidence  limits  for  MTBF  may  be  determined  by  multiplying  point  estimates  of 
MTBF  by  the  multipliers  found  in  Table  9  of  Appendix  B. 

When  actual  failure  times  are  not  known,  the  calculation  of  maximum  likelihood 
estimates  requires  a  complicated  iterative  procedure  which  can  only  be 
achieved  using  a  computer  algorithm.  In  addition,  the  estimates  are  not  as 
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accurate  as  they  would  be  if  actual  failure  times  are  known  and  used.  It  is 
important  then  to  collect  the  actual  failure  times  (in  total  test  time)  during 
development  testing.  See  Chapter  10  for  more  information  on  this  topic. 

In  Case  Studies  9-1  and  9-2,  we  demonstrate  the  procedures  for  preparing 
idealized  and  planned  growth  curves  and  for  tracking  reliability  growth. 
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CASE  STUDY  NO.  9-1 


Background 

A  new  helicopter  system  has  been  proposed.  It  is  required  to  have  a  Mean  Time 
between  Mission  Failure  (MTBMF)  of  50  hours.  Past*  experience  has  shown  that 
an  average  MTBMF  of  20  hours  can  bj  expected  during  the  initial  test  phase. 
Four  test  phases  are  planned,  and  the  manufacturer  intends  to  use  test- 
analyze-fix-test  (TAFT)  during  all  but  the  final  test  phases.  Delayed  fixes 
will  be  incorporated  at  the  end  of  all  but  the  final  test  phase. 

Determine 

1.  Construct  the  idealized  curve  for  the  program  when  the  initial  test  phase 

is  100,  200,  300  hours,  and  the  total  test  time  is  1000  hours. 

2.  Construct  an  idealized  curve  and  a  planned  growth  curve  when  the  tutal 

test  time  is  2000  hours,  and  the  four  test  phases  are  of  equal  length. 

Solutions 

l.a.  t1  =100  T  =  1,000 

Mj  =  20  Mj.  =  50 

i.  Solve  for  a  in  the  model,  using  the  approximation  9.3 

a  =  loge(100/1000)-l  *  {(lo';e(1000/100)+l)2 
+  21oge(50/20)}1/2 
=  0.267 

ii.  Determine  points  on  the  curve  using  equation  9.1 


M(t)  =  MI(t/t1)°'(l-a)"1 
t  M(t) 


<100 

20 

100 

27 

300 

36 

500 

42 

700 

46 

900 

49 

1000 

50 
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iii.  Sketch  the  curve. 


100  3  00  800  700  9  00  1000 

HOURS 


l.b.  tj  = 

200 

T  =  1000 

MI  = 

20 

30 

i.  Solve  for  a 

a  =  log  (200/ 1000) -1  +  { (log  (1000/200)+1)2 

6  c 

+  21oge(50/20)}1/2 
=  0.33 

ii.  Determine  points  on  the  curve,  using  equation  9.1 


t 

M(t) 

<200 
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iii.  Sketch  the  curve 


E 


i 


iii.  Sketch  the  curve 


U1 

i . 


=  500 

T  =  2000 

=  20 

Mf=  50 

Solve 

for  a 

a 

=  loge(500/2000)- 

-  0.356 

Determine  points  on  the 

t 

M(t) 
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iii.  Sketch  the  idealized  curve  and  superimpose  a  planned  growth  curve. 


PHASE  I 


PHASE  2 


PHASE  3 


1000 

HOURS 


Comment a i 


1.  Note,  for  the  solution  to  question  1,  how  the  length  of  the  initial  test 
phase  (t^)  affects  the  growth  parameter  a.  The  a  of  0.38  in  part  c.  may  even 
be  too  ambitious.  The  initial  test  phase  length  (t1 )  should,  however,  be  long 
enough  so  that  the  average  MTBF  of  Mj  is  achievable. 


2.  Note  that,  at  various  times  during  the  test,  the  planned  growth  curve 
either  exceeds  or  falls  below  the  idealized  curve.  The  relatively  low  values 
of  the  planned  curve  toward,  and  at  the  end  of,  the  second  test  phase  may  be 
cause  for  some  concern.  Some  fairly  substantial  increases  in  reliability  are 
required  during  the  third  test  phase  to  get  the  program  back  on  track. 


CASE  STUDY  NO.  9-2 


Background 

For  the  system  proposed  in  Case  Study  No.  9-1,  the  data  from  the  final  test 
phase  have  been  collected.  The  failure  times  as  measured  in  total  test  hours 
are  {12,  70,  105,  141,  172,  191,  245,  300,  340,  410,  490}. 

Determine 

1.  Calculate  the  MTBMF  of  the  system  at  the  end  of  the  final  test  phase. 

2.  Calculate  a  90%  lower  limit  on  the  system  MTBMF. 

Solution 

l.a.  If  we  assume  no  growth  during  this  test  phase,  the  estimated  MTBMF  is 
the  ratio  of  the  total  time  to  the  number  of  failures.  This  value  is  500/11 
or  45.4  hours. 

l.b.  If  fixes  are  being  incorporated  during  this  test  phase  as  suggested  in 
the  background  for  Case  Study  No.  9-1,  then  a  reliability  growth  analysis  is 
more  appropriate  than  one  based  upon  the  assumption  of  no  growth,  as  in  l.a. 

i.  Maximum  likelihood  estimate  for  P  is 

p  =  11/ { (.(11)  log  500)  -  (log  12+log  70+log  105+log  141+ 

6  6  6  0  6 

loge172+loge191+loge245+loge300+loge340+loge4l0+loge490)} 

P  =  0.89 

ii.  Maximum  likelihood  estimate  for  \  is 


K 


0.044 


iii.  Estimated  MTBMF  after  final  test  phase 

r(500)  =  0.0198,  and  the  estimated  MTBMF  is  the  reciprocal  of 
r(500),  which  is  50.6  hours. 


2.  Using  Appendix  B,  Table  9,  for  a  time  terminated  test,  we  find  the  lower 
90%  confidence  limit  multiplier  for  11  failures  to  be  0.565.  The  lower  limit 
is 


e  >  eL 

>  (0.565)6 

>  (0.565' (50.6) 
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>  26.6. 


We  are  90%  confident  that  the  true  MTBMF  is  at  least  28.6  hours. 

Commentary 

The  estimated  MTBMF  assuming  no  growth  is  45.4  hours.  (See  l.a.  above.)  Note 
that  this  estimate  was  computed  ■>'  'ing  the  number  of  failures  and  does  not  take 
into  account  the  actual  failure  times.  It  cannot  show  that  the  times  between 
successive  failures  seem  to  be  increasing.  If  fixes  are  being  incorporated 
during  the  test,  then  the  reliability  growth  analysis  is  more  appropriate. 
With  this  analysis,  the  estimated  MTBMF  is  50.6  hours.  The  type  of  analysis 
used,  however,  should  not  be  determined  by  the  data  but  rather  by  a  realistic 
assessment  of  the  test  program. 


CHAPTER  10 


DATA  BASE  CONSIDERATIONS 


INTRODUCTION 

Previous  chapters  of  this  text  have  preseated  descriptions  of,  and  analytical 
techniques  for,  quantitatively  assessing  system  suitability  parameters  - 
reliability,  maintainability  and  availability.  Topics  discussed  in  these 
chapters  included  sample  size,  test  hours,  test  article  configuration,  etc., 
all  of  which  formulate  the  quantitative  characteristics  of  the  data  base  which 
supports  our  assessment. 

In  contrast,  this  chapter  presents  a  discussion  of  qualitative  data  base 

characteristics.  For  example,  it  is  important  to  conduct  sufficient  testing 
on  any  prototype  system,  but  it  is  essential  that  a  production  decision  be 
supported  by  a  data  base  composed  of  production  configuration  test  data. 

Also,  a  acta  base  format  must  be  structured  before  any  actual  testing  is 

conducted  to  assure  that  the  required  information  is  collected  during  testing. 
Finally,  the  availability  of  a  meaningful  reliability  data  base,  early  in 

production,  that  reflects  early  deployment  performance  can  be  especially 
valuable  from  both  a  readiness  and  an  economic  viewpoint. 

These  and  other  qualitative  characteristics  must  be  considered  on  an  a  priori 
basis  to  assure  that  the  data  base  under  development  can  support  the  required 
assessment. 

« 

TEST  EXPOSURE  CONSIDERATIONS 

Perhaps  one  of  the  most  important  subjects  to  be  considered  in  the  evaluation 
of  RAM  characteristics  is  the  subject  of  test  exposure.  The  term  "test  ex¬ 
posure"  refers  to  the  amount  (quantity  and  quality)  of  testing  performed  on  a 
system  or  systems  in  an  ffort  to  evaluate  performance  factors.  The  connota¬ 
tion  of  the  term  test  sure  should  include  much  more  than  what  is  meant  by 
the  classical  "sample  size."  When  considering  single  shot  devices,  test 
exposure  refers  to  the  number  of  items  expended.  On  the  other  hand,  for 

non-repairable,  continuous  operation  systems,  i.e.,  destructive  testing,  test 
exposure  refers  to  the  amount  of  time  consumed  during  the  test.  In  this 

situation,  the  number  of  items  required  for  testing  is  not  known  until  the 

test  is  completed,  i.e.,  the  required  amount  of  time  on  test  has  been 

achieved.  This  results  because  the  actual  operating  life  of  each  unit  is 
unknown  until  after  the  test  is  completed. 

Now  consider  the  case  of  non-destructive  testing  on  single  shot  or  continuous 
operation  systems.  For  a  single  shot  system,  non-destructive  test  exposure 
refers  to  the  number  of  operating  cycles.  All  cycles  could,  in  theory,  be 
performed  on  a  single  item.  For  a  continuous  operation  system,  non¬ 
destructive  test  exposure  refers  to  the  amount  of  test  time  to  be  accumulated 
just  as  for  the  destructive  testing  case.  However,  with  non-destructive 


testing,  the  test  designer  should  exercise  good  judgment  in  precisely  defining 
the  test  exposure.  Elements  to  consider  are: 

-  Should  the  time  required  be  accumulated  on  one  system  or  several 
systems? 

-  Should  prototypes  or  production  models  be  used? 

-  Should  testing  be  accelerated  by  eliminating  nonoperating  time?  If 
so,  what  is  the  effect? 

-  Do  we  anticipate  changes  in  equipment  failure  rate  due  to  age  effects 
or  design  modifications? 

-  Is  the  external  environment  commensurate  with  the  requirements? 

One  System  Vs.  Several  Systems 

Testing  one  item  for  100  hours  is,  practically  speaking,  not  the  same  as  test¬ 
ing  ten  items  for  10  hours  each,  although  when  considering  the  exponential 
model,  the  two  tests  are  theoretically  equivalent.  The  major  statistical 
assumptions  involved  are  a  constant  failure  rate  and  a  homogeneous  population. 
Although  the  two  test  alternatives  presented  are  not  equivalent  for  practical 
evaluation  purposes,  it  is  not'  a  case  of  one  being  "better"  than  the  other. 
Each  alternative  has  a  desirable  feature.  The  test  involving  ten  items  has 
the  advantage  of  using  a  greater  cross-section  of  the  population.  This  is 
particularly  important  if  the  population  quality  is  inconsistent.  On  the 
other  hand,  the  test  of  one  item  for  100  hours  has  the  advantage  of  exploring 
more  fully  the  effects  of  equipment  age  on  system  reliability. 

As  a  general  rule,  for  evaluation  purposes,  it  is  desirable  to  test  a  "mod¬ 
erate"  number  of  items  for  a  "moderate"  period.  This  makes  the  test  rela¬ 
tively  insensitive  to  the  underlying  statistical  assumptions  of  constant 
failure  rate  and  sample  homogeneity.  One  compromise  between  sample  size  and 
test  exposure  requires  that  a  minimum  of  three  items  will  operate  for  at  least 
1.5  times  the  minimum  acceptable  value  (MAV).  As  another  example,  MIL- 
STD  781C  recommends  for  production  acceptance  testing  that  10%  of  the  lot  be 
tested,  down  to  a  minimum  of  3  items,  and  up  to  a  maximum  of  20  items. 

Another  recommendation  presented  in  MIL-STD  781C  is  for  each  test  article  to 
operate  at  least  one  half  the  average  operating  time  of  all  articles  on  test. 
If  some  of  the  test  articles  experience  an  excessive  number  of  failures  there 
is  a  natural  tendency  for  them  to  accumulate  little  test  exposure,  simply 
because  of  the  difficulty  of  keeping  them  on  test.  A  constraint  of  this  type 
should  minimize  this  biasing  tendency. 

Accelerated  Testing  and  Extei ' *1  Environment 

Because  the  operating  life  of  most  systems  generally  exceeds  the  available 
test  period  length,  some  form  of  accelerated  testing  is  often  performed.  The 
acceleration  may  consist  of  merely  eliminating  standby  time  from  the  duty 
cycle  to  subjecting  the  equipment  to  some  sort  of  overstress  conditions.  The 
evaluator  should  be  aware  of  the  impact  of  accelerated  testing  on  the  equip¬ 
ment  and  how  it  will  influence  his  analysis. 


Circumstances  not  defined  in  the  mission  scenario  can  significantly  impact  the 
results  of  a  reliability  test  program.  Every  effort  should  be  made  to  control 
these  and  other  effects  so  that  the  test  environment  is  commensurate  with  the 
intended  operational  environment. 

Prototypes  Vs .  Production  Models 

In  most  cases,  there  is  no  choice  in  this  matter.  For  instance,  in  develop¬ 
ment  testing  there  are  generally  only  prototype  models  available.  However, 
for  operational  testing  and  evaluation,  production  models  should  be  used.  In 
this  case,  we  are  trying  to  evaluate  the  "final"  configuration  system  as  it 
will  actually  perform  "in  the  field,"  rather  than  evaluating  the  system's 
inherent  capabilities. 

COMPOSING  THE  RELIABILITY  DATA  BASE 

When  we  use  data  analysis  techniques  that  consider  the  possibility  of  a  chang¬ 
ing  failure  rate,  we  are  acknowledging  that  there  is  reason  to  suspect  that 
the  failure  rate  may  not  be  constant.  Two  of  the  most  common  causes  for  a 
changing  failure  rate  are: 

-  Inherent  changes  in  the  equipment  as  it  accumulates  more  hours  of 
operation,  i.e.,  as  it  ages. 

-  Changes  in  the  equipment  due  to  design  changes. 

There  is,  at  present,  no  readily  usable  statistical  technique  for  analyzing 
system  reliability  which  is  affected  by  two  or  more  of  these  factors.  We 
cannot  define  a  precise  method  for  evaluating  test  results  which  are  derived 
from  systems  which  are  improving  as  a  consequence  of  design  modifications  and 
at  the  same  time  degrading  as  a  consequence  of  wear-out.  The  only  guidance 
for  this  situation  is  to  perform  individual  analyses  on  each  of  the  subsystems 
for  that  period  of  time  when  they  have  a  fixed  configuration.  Total  system 
reliability  can  be  obtained  by  piecing  together  the  subsystem  reliabilities  in 
accordance  with  a  system  reliability  model  (series,  parallel,  etc.').  See 
Chapter  2  for  details  on  the  application  of  this  technique.  See  Case  Study 
10-1. 

Age-Dependent  Analysis 

When  the  configurations  of  the  systems  on  test  are  the  same  and  fixed,  we  may 
be  interested  in  observing  the  effects  of  aging  on  the  failure  character¬ 
istics.  For  this  situation,  we  are  required  to  record  the  actual  age  of  the 
system  when  it  has  failed.  Each  element  of  the  data  base  represents  the  age 
of  the  failed  system  at  the  time  of  failure. 

As  an  example,  suppose  that  3  systems  have  been  on  test.  System  1  operates 
from  0  to  1000  hours  and  failed  3  times.  The  times  of  failure  were  20  hours, 
90  hours,  and  615  hours.  System  2  operated  from  100  to  800  hours  and  failed  4 
times.  The  times  of  failure  were  130  hours,  195  hours,  345  hours,  and  520 
hours.  System  3  operated  from  500  to  1000  hours  and  failed  2  times.  The 
times  of  failure  were  560  hours  and  820  hours .  The  recorded  times  represent 
the  age  of  the  system  at  the  time  of  failure.  For  an  age-dependent  analysis, 
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....... . 


the  cumulative  operating  times  are  actually  irrelevant.  The  data  base  for 
this  test  is  the  set  of  failure  times. 


{20,  90,  130,  195,  345,  520,  560,  615,  820} 


See  Figure  10-1  for  a  graphical  portrayal  of  composing  this  type  of  data  base. 


FIGURE  10  - 1  SYSTEM  FAILURE  TIMES 


BRACKETS  INDICATE  WNEN  A  SYSTEM  WAS  ON  TEST. 


See  Case  Study  10-2  for  another  example  of  composing  the  data  base  for  an 
age-dependent  analysis. 

Growth  Analysis 

When  a  system  or  systems  are  undergoing  a  development  type  test  during  which 
design  modifications  are  being  incorporated,  we  may  be  interested  in  observing 
the  effects  of  these  design  changes  on  the  reliability  of  the  system.  In  this 
situation,  the  systems  are  being  tested  so  that  weaknesses  in  design  will  sur¬ 
face  as  failures.  Ideally,  when  a  failure  occurs,  all  testing  will  stop  while 
the  failure  is  analyzed  and  a  design  modification  is  developed.  The  modifica¬ 
tion  is  incorporated  on  all  test  systems  and  testing  is  resumed  until  the  next 
failure  occurs .  Theoretically,  for  this  type  of  testing,  we  are  not  inter¬ 
ested  in  the  age  of  each  of  the  systems .  Rather,  we  are  interested  in  the 
cumulative  time  they  have  been  on  test  when  a  failure  occurs .  Each  element  of 
the  data  base  for  a  reliability  growth  analysis  represents  the  total  test  time 
accumulated  by  all  systems  at  the  exact  time  a  failure  occurs. 

As  an  example,  suppose  that  3  systems  have  been  engaged  in  development  test¬ 
ing.  In  Figure  10-2  we  display  the  failure  patterns  of  the  systems. 


FIGURE  10-2  SYSTEM  TIME  ON  TEST 


SYSTEM  I 

SYSTEM  2 

SYSTEM  3 


BRACKETS  INDICATE  WHEN  A  SYSTEM  WAS  ON  TEST. 

NUMBERS  IN  PARENTHESES  REPRESENT  THE  CHRONOLOGICAL  OROER  OF 
FAILURES. 


System  1  was  the  only  one  on  test  for  the  first  50  hours.  System  2  began 
operating  at  the  50-hour  point  and  System  3  at  the  150-hour  point.  System  2 
was  taken  off  test  at  the  240-hour  point  and  returned  to  testing  at  the  400- 
hour  point.  System  1  was  taken  off  test  at  the  330-hour  point  and  did  not 
return.  To  compose  the  data  base  for  this  test  we  must  determine  how  much 
test  operating  time  has  been  accumulated  when  a  failure  has  occurred. 


FIGURE  10-3.  CUMULATIVE  TEST  EXPOSURE/FAILURE  HISTORY 
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The  data  base  for  this  test  is  the  set  of  failure  times: 

{40,  100,  150,  400,  520,  660,  830}. 


Note  that  these  failure  times  have  nothing  to  do  with  the  system  ages.  In 
fact,  the  data  base  is  the  same  whether  the  systems  have  no  operating  time  on 
them  at  the  start  of  the  test  or  the  systems  have  substantial  amounts  of 
operating  time  at  the  start  of  the  test.  See  Case  Study  10-2  for  another 
example  of  composing  a  reliability  growth  data  base. 


COMBINING  DT  AND  OT  DATA 


One  method  of  obtaining  a  relatively  larger  data  base  at  a  given  time  in 
development  is  to  combine  data  derived  from  both  Developmental  (DT)  and  Oper¬ 
ational  (OT)  testing  into  a  single,  larger  "homogeneous"  data  base.  All  DoD 
components  have  utilized  this  concept,  and  expanded  use  of  the  aggregated  data 
base  concept  is  anticipated  as  a  direct  consequence  of  DoD  policy  to  execute 
shorter  acquisition  cycles  supported  by  comprehensive  test  programs. 

The  advantages  of  a  larger  data  base  are  clear.  It  provides  the  analyst  the 
information  necessary  to  more  accurately  assess  system  characteristics  and, 
simultaneously,  to  be  more  confident  of  his  results.  Likewise,  the  pitfalls 
of  larger  "aggregate"  data  bases  should  be  apparent.  All  the  assumptions 
inherent  in  the  various  system  math  models  apply  to  the  data  base  elements. 
These  assumptions  relate  to  failure  rate,  test  article  configuration,  sample 
size,  test  environment  realism,  etc.  Any  DT  or  OT  data  point  that  is  placed 
in  an  aggregate  data  base  must  adhere  to  the  applicable  assumptions.  There¬ 
fore,  from  a  practical  viewpoint,  the  development  of  a  meaningful  aggregate 
data  base  requires  that  specific  ground  rules  be  established  prior  to  data 
collection  and  data  base  development.  Further,  the  developer,  user  and  test 
organization  must  jointly  review  available  data  and,  using  the  established 
ground  rules,  decide  what  data  to  place  in  an  aggregate  data  base.  One  ac¬ 
cepted  alternative  to  a  single  category  data  base  is  the  compartmented  ag¬ 
gregate  data  base.  Under  this  concept,  various  data  compartments  are  estab¬ 
lished  within  the  aggregate  data  base.  One  compartment,  for  example,  would 
contain  DT  and  OT  data  which  could  reasonably  support  the  assessment  of  the 
entire  system's  performance.  Another  compartment's  data  would  only  be  used  to 
analyze  the  performance  of  specific  subsystems,  whose  configuration  may  have 
changed  during  the  final  stages  of  system  development. 

The  key  factor  in  developing  an  aggregate  DT/OT  data  base  is  the  homogeneity 
and  applicability  of  the  test  data.  That  is,  does  the  larger  aggregate  data 
base  contain  information  which  supports  meaningful  performance  assessments  of 
the  present  system,  or  subsystems,  when  operated  in  accordance  with  current 
mission  scenario  requirements?  If  not,  an  assessment  based  on  this  data  will 
lead  to  erroneous  conclusions . 

EARLY  DEPLOYMENT  DATA 

One  final  aspect  of  data  base  development  is  the  timely  availability  of  data 
required  to  accurately  evaluate  system  suitability  characteristics.  It  is 
especially  important  to  obtain  timely  access  to  information  which  describes 
malfunctions  or  "shortcomings"  which  have  been  observed  during  early  deploy¬ 
ment.  The  availability  of  this  information  permits  the  developer  and  user  to 
define  and  implement  hardware,  software,  and  training  modifications  at  minimal 
retrofit  cost  and  in  the  shortest  time.  The  two  methods  described  below  have 

been  utilized  to  obtain  the  desired  information  at  the  earliest  time  possible. 

* 

Early  Field  Data  Retrieval  System 

The  combination  of  an  on-site  developer  representative  and  a  structured  field- 
data  retrieval  system  has  been  used  successfully  to  obtain  timely  performance 
information  during  initial  deployment.  The  developer's  representative  is 
responsible  for  assuring  that  each  malfunction  or  incident  is  accurately 


reported  on  a  real-time  basis.  Information  is  collected  and  analyzed  to 
determine  the  presence  of  failure  trends  which  require  prompt  resolution.  The 
essential  feature  of  this  concept  is  the  real-time  reporting  of  data  by  a 
trained  observer  to  the  developer  who  can  initiate  appropriate  action.  Normal 
service  maintenance  reporting  systems  do  not  provide  the  required  response 
time  nor  the  accurate,  detailed  information. 

Lead-The-Force  Program 

A  Lead-The-Force  (LTF)  program,  sometimes  referred  to  as  Lead-The-Fleet,  has 
one  primary  objective.  That  is,  to  obtain,  at  the  earliest  possible  time,  in¬ 
formation  on  failure  and/or  wear-out  modes  that  all  units  in  the  force  (fleet) 
are  likely  to  experience  at  some  future  time.  Of  specific  interest  are  those 
failure  or  wear-out  modes  which  are  unexpected,  preit^ture,  very  costly,  or 
combinations  thereof. 

For  example,  it  would  be  cost  effective  to  determine,  early  in  production, 
that  the  wing  structure  of  a  new  cargo  aircraft  was  experiencing  premature 
fatigue,  failure,  or  that  a  new  tank  engine  was  subject  to  an  unexpected 
failure  mode  that  went  undiscovered  during  testing. 

An  LTF  program  provides  the  desired  information  by  means  of  accelerated  usage 
of  a  few  systems  and  by  simultaneous  in  depth  reporting  of  all  malfunctions 
and  "incidents." 

An  example  of  an  LTF  program  will  prove  helpful.  Suppose  we  are  starting  to 
deploy  a  new  light-armored  vehicle  and  the  engine/drive  train  subsystem  is 
expected  to  have  a  usable  service  life  of  60,000  miles.  During  training,  the 
average  vehicle  will  be  driven  300  miles  per  month.  At  this  rate  it  would 
require  over  16  years  before  the  first  vehicle  drive  trains  will  require  depot 
maintenance.  An  LTF  program  for  the  drive  train  subsystem  would  require  that 
a  selected  number  of  production  configuration  vehicles  accumulate  mileage 
which  is  at  least  3  to  6  times  the  average.  This  added  mileage  must  be  ac¬ 
cumulated  under  typical  conditions  of  speed,  terrain,  maintenance,  etc. 
Assuming  the  planned  LTF  program  is  executed  at  4  times  the  average  mileage, 
we  will  be  able  to  obtain  a  preliminary  view,  of  the  fleet's  drive  train  life 
cycle  performance  in  just  over  four  years.  This  information  is  valuable 
because:  it  permits  us  to  more  accurately  plan  future  depot  maintenance  re¬ 
quirements;  we  are  able  to  predict  the  occurrence  of  major  mechanical  problems 
that  could  potentially  affect  fleet-wide  mission  readiness;  we  can  start  now 
to  implement  changes  in  production  models  and  to  manufacture  retrofit  kits  or 
spares  which  can  be  installed  because  major  breakdowns  occur. 

In  summary,  an  LTF  data  base  provides  information  required  to  assess  long  term 
system  reliability  and  maintainability  characteristics  in  a  significantly 
shortened  time  period.  The  primary  benefits  are  linked  to  our  ability  to 
execute  timely,  cost  effective  actions  which  maintain  desired  readiness 
levels . 
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CASE  STUDY  NO.  10-1 


Background 

A  system  is  composed  of  four  primary  subsystems.  The  system  has  been  undergo¬ 
ing  development  testing  during  which  time  two  of  the  subsystems  have  received 
substantial  design  modifications.  The  other  two  subsystems  are  relatively 
mature  and  have  not  been  modified.  One  of  these  subsystems  has  experienced 
wearout  failures;  the  other,  has  not.  The  following  information  on  the  sub¬ 
system  is  available. 

Subsystem  Information 

1&2  Modifications  incorporated 

MTBF  estimates  0^  =  450  hours 
@2  =  200  hours 

3  Wearout  failures 

MTBF  estimate  0^  =  300  hours 

4  No  wear  out  failures 

MTBF  estimate  0^  =  800  hours 


Determine 

1.  If  the  subsystems  are  in  series,  what  is  system  mission  reliability  for  a 
100-hour  mission? 

2.  If  subsystems  1  and  2  are  in  parallel  (active  redundancy),  what  is  system 
mission  reliability  for  a  200-hour  .mission? 

Solution 


In  order  to  determine  system  reliability,  we  first  compute  subsystem  reli¬ 
ability  using  the  reliability  function, 

R(t)  =  e't/6. 

Reliability 


Subsystem 

100-Hour 

200-Hour 

1 

e-l°°/«°  =  0.80 

e-200/450  .  0-64 

2 

e-100/200  =  „  61 

e-200/200  .  „  3? 

3 

e-100/300  ,  „  „ 

e-200/300  .  „  M 

4 

e-100/800  „  0.86 

e‘200/800  =  0^78 
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1.  Since  the  subsystems  are  all  in  series,  the  system  reliability  is 


(0.80) (0.61) (0 . 72) (0 .88)  =  0.31. 


2.  Since  subsystems  1  and  2  are  in  parallel  and  then  together  are  in  series 
with  sybsystems  3  and  4,  the  system  reliability  is 


See  Equations 


n 

R  * 
system 

1  -  n  (1  -  R.) 

i=l 

x  R, 

1-n 

series 

—  — 

parallel  systems 

systems  > 

[1  -  (1  -  0.64) (1  -  0-37) ) (0 .51) (0. 78) 
=  [1  -  (0 . 36) (0.63) ] (0.51) (0 .78) 

=  (1  -  0.23)(0.51)(0.78) 

=  (0.77) (0.51) (0. 78)  =  0.31. 


Commentary 

In  the  background,  we  have  given  the  MTBF  estimates  for  each  subsystem.  In 
order  to  obtain  such  estimates,  analysis  should  be  done  on  failure  data  for 
each  subsystem.  In  particular,  a  reliability  growth  analysis  should  be  per¬ 
formed  on  subsystems  1  and  2.  (See  Chapter  9  for  details  on  this  analysis.) 
For  subsystem  4,  an  exponential  analysis  would  be  appropriate.  (See  Chapter  7 
for  details  on  this  analysis.)  An  analysis  using  a  non-homogeneous  Poisson 
process  would  no  doubt  be  necessary  for  subsystem  3.  (See  Chapter  7  for  a 
description  of  when  this  type  of  analysis  is  required.)  The  primary  purpose 
of  this  case  study  is  to  show  how  the  concepts  in  Chapter  2  can  be  used  to 
obtain  an  estimate  of  system  reliability  using  estimates  of  subsystem  reli¬ 
ability.  The  essential  utility  of  this  procedure  is  for  the  situation  where 
different  methods  of  analysis  for  different  subsystems  are  necessary.  The 
fact  that  the  two  system  reliabilities  are  0.31  is  coincidental. 


CASE  STUDY  NO.  10-2 


Determine 

1.  Compose  the  data  base  for  an  age-dependent  analysis. 

2.  Compose  the  data  base  for  a  growth  analysis. 

Solution 

1.  For  an  age-dependent  analysis,  failure  times  for  each  system  are  recorded 
as  system  age  at  the  time  of  failure. 

System  1  failure  times:  600,  800,  1150,  1400 

System  2,  failure  times:  150,  650 

System  3  failure  times:  75,  300,  500 

System  4  failure  times:  400,  550 

The  data  base  is  the  set: 

{75,  150,  300,  400,  500,  550,  600,  650,  800,  1150,  1400}. 
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2.  For  a  growth  analysis,  failure  times  for  each  system  are  recorded  as 
accumulated  test  exposure  at  the  time  of  failure. 


Failure 

System  Test  Hours 

Accumulated 

Number 

1 

2 

3 

4 

Test  Time 

1 

50 

50 

0 

0 

100 

2 

100 

100 

0 

0 

200 

3 

175 

175 

75 

0 

425 

4 

300 

300 

200 

0 

800 

5 

400 

400 

300 

100 

1200 

6 

550 

550 

450 

250 

1800 

7 

600 

550 

500 

300 

1950 

8 

650 

550 

500 

350 

2050 

9 

700 

550 

500 

400 

2150 

10 

850 

550 

500 

550 

2450  * 

11 

900 

550 

550 

600 

2600 

* 

The  data  base  is  the  set: 

4 

{100,  200,  425,  800,  1200,  1800,  1950,  2050,  2150,  2450,  2600}. 
Commentary  A- 

For  comparative  purposes,  this  case  study  composes  the  data  base  two  ways:  as 
a  function  of  age,  which  is  appropriate  for  a  fixed-coDfiguration  test;  and  as 
a  function  of  accumulated  test  exposure,  whicfi  is  appropriate  for  a  test  with 
design  changes.  It  must  be  emphasized  that  this  is  done  for  illustrative 
purposes  only.  An  actual  data  base  is  composed  one  way  or  the  other  depending 
on  the  nature  of  the  test. 


a 
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NORMAL  APPROXIMATION  TO  BINOMIAL 


When  p,  the  probability  of  failure  on  a  given  trial,  is  moderate  (0.2  <  p  < 
0.8)  and  n  the  number  of  trials  is  large  (n  >  30),  the  normal  distribution 
provides  reasonable  approximations  to  binomial  probabilities.  This  approxima¬ 
tion  is  detailed  b^iow.  Note  that  Z  is  the  notation  for  the  standard  normal 
variable.  (See  Appendix  B,  Table  2.) 

The  probability  of  k  or  fewer  failures  out  of  n  trials  is  approximately  equal 
to 

1  -  P(Z  >  (k+0.5-np)/Vnp(l-p)). 

The  probability  of  at  least  k  failures  out  of  n  trials  is  approximately  equal 
to 

P(Z  >  (k-0.5-np)/Vnp(l-p)). 

The  probability  of  between  kj  and  failures  out  of  n  trials  inclusive  is 
approximately  equal  to 

P(Z  >  (k1-0.5-np)/Vnp(l-p))  -  P(Z  >  (k2+0.5-np)/Vnp(l-p)) 

We  have  listed  the  approximations  in  the  form 
P(Z  >  a) 

so  that  the  use  of  Appendix  B,  Table  2  is  direct. 

As  an  example,  suppose  that  n  =  40  and  p  =  0.3.  The  probability  of  between  10 
and  20  failures  inclusive  is 

P(Z  >  (10-0. 5-(40)(0.3)/V(40) (0.3) (0.7)) 

-  P(Z  >  (20+0.5-(40)(0.3)/VT40)(0.3)(0.7T)  . 

Simplifying  we  obtain 

P(Z  >  -  0.86)  -  P(Z  >  2.93). 

Now  from  Appendix  B,  Table  1  ,  we  find  that  P(Z  >  -  0.8u)  =  0.8051  and  P(Z  > 
2.93)  =  0.0017.  Consequently',  the  probability  that  between  10  and  20  failures 
inclusive  occur  is  approximately  0.8034. 

The  value  using  a  set  of  binomial  tables  is  0.8017. 
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A-2.  POISSON  APPROXIMATION  TO  BINOMIAL 


When  p,  the  probability  of  failure  on  a  given  trial,  is  extreme  (p  <  0.2  or  p 
>  0.8)  and  n,  the  number  of  trails,  is  large  (n  >  30),  the  Poisson  distribu¬ 
tion  provides  reasonable  approximations  to  binomial  probabilities.  We  make 
the  identification  m  =  np  and  use  Poisson  tables  to  determine  the  prob¬ 
abilities  of  events  in  a  binomial  experiment. 

As  an  example,  suppose  that  n  =  40  and  p  =  0.05,  so  that  m  =  40(0.05)  =  2. 
The  probability  of  between  5  and  10  failures  is  the  difference  between  the 
probability  of  10  or  fewer  failures  (1.000)  and  the  probability  of  4  or  fewer 
failures  (0.947).  (See  Appendix  B,  Table  3.)  The  difference  is  0.053.  Using 
a  set  of  binomial  tables  we  obtain  0.0480. 


A-3.  NORMAL  APPROXIMATION  TO  POISSON 


When  the  product  AT  is  greater  than  5 ,  the  normal  distribution  provides 
reasonable  approximations  to  Poisson  probabilities.  The  approximation  is 
detailed  below.  Note  that  Z  is  the  notation  for  the  standard  normal  variable. 
(See  Appendix  B,  Table  2) 

The  probability  of  k  or  fewer  failures  during  time  T  is  approximately 
1  -  P(Z  >  (k+0.5-\T)/VAT)  . 

The  probability  of  at  least  k  failures  during  time  T  is  approximately 
P(Z  >  (k-0.5-\T)A/AT)  . 

The  probability  of  between  k-  and  k^  failures  inclusive  during  time  T  is 
approximately  " 

P(Z  >  (kj-O.S-ATJ/VAT)  -  P(Z  >  (k2+0 .5-AT)/VAT )  .. 

We  have  listed  the  approximations  in  the  form 
P(Z  >  a) 

so  that  the  use  of  Appendix  B,  Table  2  is  direct. 

As  an  example,  suppose  that  the  failure  rate  A  is  0.01  and  the  test  time  T  is 
1000  hours.  The  probability  of  between  8  and  15  failures  inclusive  is 

p(z  >  (8-o.5-(o.oi)(igoo)/V(o7oi)"(io"ooT) 

-  P(Z  >  , 15+0. 5-(0.01)(1000)/V(0. 01) (1000))  . 

The  above  expression  reduces  to 
P(Z  >  -0.79)  -  P(Z  :  1.74). 

Now  P(Z  >  -0.79)  =  0.7852  and  P(Z  >  1.74)  =  0.049,  so  the  probability  that 
between  8  and  15  failures  inclusive  occur  is  approximately  0.7443. 

Using  the  Poisson  tables  (Appendix  B,  Table  3),  we  obtain  the  probability  more 
precisely  as  0.731. 
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TABLE  1 


BINOMIAL  TABLES 

Table  gives  probability  of  c  or  fewer  failures. 

NOTE:  Only  excerpts  of  binomial  tables  required 

to  work  case  studies  are  provided  in  Table  1 
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.9633 

.0367 

2.29 

.9890 

.0110 

2.79 

.9974 

.0026 

1.80 

.9641 

.0359 

2.30 

.9893 

.0107 

2.80 

.9974 

.0026 

1.81 

.9649 

.0351 

2.31 

.9896 

.0104 

2.81 

.0975 

.0025 

1.8’. 

.9656 

.0344 

3.32 

.9898 

.0102 

2.82 

.9976 

.0024 

1.83 

.9664 

.0336 

2.33 

.9901 

.0090 

2.83 

.9977 

.0023 

1.84 

.9671 

.0329 

2.34 

.9904 

.0096 

2.84 

.9977 

.0023 

1.86 

.9678 

.0322 

2.35 

.9906 

.0094 

2.85 

.9978 

.0022 

1.86 

.9686 

.0314 

2.36 

.9909 

.0091' 

2.86 

.9079 

.0031 

1.87 

.9693 

.0307 

2.37 

:9911 

.0089 

2.87 

.9079 

.0021 

1.88 

.9699 

.0301 

2.38 

.9913 

.0087 

2.88 

.9080 

.  0020 

1.89 

.9706 

.0294 

2.39 

.9916 

.0084 

2.89 

.9081 

i 

.0019 

1.90 

.9713 

.0287 

2.40 

.9918 

.0082 

2.90 

.9081 

.0019 

1.91 

.9719 

.0281 

2.41 

.9920 

.0080 

2.91 

.0082 

.0018 

1.92 

.9726 

.0274 

2.42 

.9922 

.0078 

2.92 

.0083 

.0017 

1.93 

.9732 

.0268 

2.43 

.9925 

.0075 

2.93 

.0083 

.0017 

1.94 

.9738 

.0282 

2.44 

.9927 

.0073 

2.94 

.0084 

.0016 

1.95 

.9744 

.0256 

2  45 

.9929 

.0071 

2.95 

.9084 

.0016 

1.96 

.9750 

.0250 

2.46 

.9931 

.0060 

2.96 

.9085 

.0015 

1.97 

.9756 

.0244 

2.47 

.9932 

.0068 

2.97 

.9085 

.0015 

1.98 

.9761 

.0239 

2.48 

.9934 

.0066 

2.98 

.9086 

.0014 

1.99 

.9767 

.0233 

2.49 

.9936 

.0064 

2.90 

.9086 

.0014 

2.00 

.9773 

.0237 

2.50 

.9938 

.0062 

3.00 

.9087 

.0013 

TABLE  3 


CUMULATIVE  POlSSON  PROBABILITIES 


Description  and  In» traction* 


This  table  lists  value*  of 

G.  -(e)  *  X  *77—  ,  ■  *•  M. 

A’T  ixO 

The  table  it  indexed  horizontally  by  c  and  vertically 

by  o,  and  the  tabled  probabilities  are  rounded  to  the 

nearest  0.001.  See  Chapter  5,  section  entitled 
"Poisson  Model"  for  a  discussion  of  the  Poisson  dis* 
tribution  function,  ^(c). 

Example  1.  Assume  a  Poisson  model  where  the  failure 
rate  (A)  is  0.016  and  the  time  (T)  is  200  hours. 

Thus,  a  =  AT  ■  (0.016) (200)  *  3.2.  To  find  the  prob¬ 

ability  of  6  or  fewer  failures  occurring,  we  let 
a  *  3.2,  c  *  5  and  read  the  value  directly  from  the 
table  as  0.895. 

Example  2.  The  probability  of  exactly  c  failures 
occurring  is 

P(c  or  fewer  failures) 

-  P(c-1  or  fewer  failures). 

For  a  =  3.2  and  c  -  5.  this  probability  is 

0.895  -  0.781  *  0. 111. 

Note  that  this  example  shows  how  to  obtain  values  of 
the  function  T(c)  as  defined  in  Chapter  5. 


Example  3.  The  probability  of  anre  than  c  failures 
occurring  is 

1  •  P(c  or  fewer  failures). 

For  m  =  3.2  and  c  *  5,  this  probability  is 
1  -  0.895  =  0.105. 


0 

1 

2 

3 

4 

5 

0.01 

.990 

TToo 

0.02 

.980 

1.  000 

0.03 

.970 

1.  000 

0.04 

.961 

.999 

1.000 

0.05 

.951 

.999 

1.000 

0.06 

.942 

.998 

1.000 

0.07 

.932 

.998 

1.000 

0.06 

.923 

.997 

1.000 

0.09 

.914 

.996 

1.000 

0.10 

.905 

.995 

1.000 

0.12 

.887 

.993 

1.000 

0.14 

.869 

.991 

1.000 

0.16 

.852 

.988 

.999 

1.000 

0.18 

.835 

.986 

.999 

1.000 

0.20 

.819 

.982 

.999 

1. 000 

0.22 

.803 

.979 

.998 

1.000 

0.24 

.787 

.975 

.998 

1.000 

0.26 

.771 

.972 

.998 

1.000 

0.28 

.756 

.967 

.997 

1.000 

0.30 

.741 

.963 

.996 

1.000 

0.32 

.726 

.959 

.996 

1.000 

0.34 

.712 

.954 

.995 

1.000 

0.36 

.698 

.949 

.994 

.999 

1.  000 

0.38 

.684 

.944 

.993 

.999 

1.000 

0.40 

.670 

.938 

.992 

.999 

1.000 

0.42 

.657 

.933 

.991 

.999 

1.  000 

0.44 

.644 

.927 

.990 

.999 

1.  000 

0.46 

.631 

.922 

.988 

.999 

1.000 

0.48 

.619 

.916 

.987 

.998 

1.  000 

0.50 

.607 

.910 

.986 

.998 

1.000 

0.52 

.595 

.904 

.984 

.998 

1.000 

0.54 

.583 

.  897 

.982 

.998 

1.  000 

0.56 

.571 

.891 

.981 

.997 

1.000 

0.58 

.560 

.  885 

.979 

.997 

1.000 

0.60 

.549 

.878 

.977 

.997 

1.  000 

0.62 

.538 

.871 

.975 

.996 

1.  000 

0.64 

.527 

.  865 

.973 

.996 

.999 

0.66 

.517 

.  858 

.971 

.995 

.999 

1. 000 

0.68 

.507 

.851 

.968 

.995 

.999 

1.000 

0.70 

.497 

.844 

.966 

.994 

.999 

1.000 

0.72 

.487 

.837 

.963 

.994 

.999 

1.000 

0.74 

.477 

.830 

.961 

.993 

.999 

1.000 

0.76 

.468 

.823 

.958 

.992 

.999 

1. 000 

0.78 

.458 

.816 

.955 

.992 

.999 

1.000 

0.  8C 

.449 

.809 

.  953 

.991 

.  999 

1.000 
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TABLE  3.  CUMULATIVE  POISSON  PROBABILITIES  (CONT.) 


n 

2 

3 

5 

6 

0.82 

.440 

.802 

.950 

.990 

.998 

1. 000 

0.34 

.432 

.794 

.947 

.989 

.998 

1.000 

0.86 

.423 

.787 

ClA  A 

•  w  *  » 

.988 

.998 

1.000 

0.88 

.415 

.780 

.940 

.988 

.998 

1.000 

0.90 

.407 

.772 

.937 

.987 

.998 

1.000 

0.92 

.399 

.765 

.934 

.986 

.997 

1.000 

0.94 

.391 

.758 

.930 

.984 

.997 

1.000 

0.96 

.383 

.750 

.927 

.983 

.997 

1.000 

0.98 

.375 

.743 

.923 

.982 

.997 

.999 

1.000 

1.00 

.368 

.736 

.920 

.981 

.996 

.999 

1.000 

1.10 

.333 

.699 

.900 

.974 

.995 

.999 

1.000 

1.20 

.301 

.663 

.879 

.966 

.992 

.998 

1.000 

1.30 

.273 

.627 

.857 

.957 

.989 

.998 

1.000 

1.40 

.247 

.592 

.833 

.946 

.986 

.997 

.999 

1.  50 

.223 

.558 

.809 

.934 

.981 

.996 

.999 

1.60 

.202 

.525 

.783 

.921 

.976 

.994 

.999 

1.70 

.183 

.493 

.757 

.907 

.970 

.992 

.998 

1.80 

.165 

.463 

.731 

.891 

.964 

.990 

.997 

1.90 

.160 

.434 

.704 

.875 

.956 

.987 

.997 

2.00 

.135 

.406 

.677 

.857 

.947 

.983 

.996 

2.20 

.111 

.355 

.623 

.819 

.923 

.975 

.993 

2.40 

.091 

.308 

.570 

.779 

.904 

.964 

.988 

2.60 

.074 

.267 

.518 

.736 

.877 

.951 

.983 

2. 80 

.061 

.231 

.469 

.692 

.848 

.935 

.976 

3.00 

.050 

.199 

.423 

.647 

.815 

.916 

.966 

3.20 

.041 

.171 

.380 

.603 

.781 

.895 

.955 

3.40 

.033 

.147 

.340 

.558 

.744 

.871 

.942 

3.60 

.027 

.126 

.303 

.515 

.706 

.844 

.927 

3.  80 

.022 

.107 

.269 

.473 

.668 

.816 

.909 

4.00 

.018 

.092 

.238 

.433 

.629 

.785 

.889 

4.20 

.015 

.078 

.210 

.395 

.590 

.753 

.867 

4.40 

.012 

.066 

.185 

.359 

.551 

.720 

QAA 

•  Oti 

4.60 

.010 

.056 

.163 

.326 

.513 

.686 

.818 

4.80 

.008 

.048 

.143 

.294 

.478 

.651 

.791 

5. 00 

.007 

.040 

.125 

.265 

.440 

.616 

.762 

5.20 

.006 

.034 

.109 

.238 

.406 

.581 

.732 

5.40 

.005 

.029 

.095 

.213 

.373 

.546 

.702 

5.60 

.004 

.024 

.082 

.191 

.342 

.512 

.670 

5.  80 

.003 

.021 

.072 

.170 

.313 

.478 

.638 

6.00 

.002 

.017 

.062 

.151 

.285 

.446 

.606 

6.20 

.002 

.015 

.054 

.134 

.259 

.414 

.574 

6.40 

.002 

.012 

.046 

.119 

.235 

.384 

.542 

6.60 

.001 

.010 

.040 

.105 

.213 

.355 

.511 

6.80 

.001 

.009 

.034 

.093 

.192 

.327 

.480 

7.00 

.001 

.007 

.030 

.082 

.173 

.301 

.450 

7.20 

.001 

.006 

.025 

.072 

.156 

.276 

.420 

7.40 

.001 

.005 

.022 

.063 

.140 

.253 

.392 

7.60 

.001 

.004 

.019 

.055 

.125 

.231 

.365 

7.80 

.000 

.004 

.016 

00 

o 

• 

.112 

.210 

.338 

8. 00 

.000 

.003 

.014 

.042 

.100 

.191 

.313 

7 


1.  000 

1.000 

1.  000 
1.  000 
.999 
.999 
.999 

.998 

.997 

.995 

.992 

.988 

.983 

.977 

.969 

.960 

.949 

.936 

.921 

.905 

.887 

.867 

.345 

.322 

.797 

.771 

.744 

.716 

.687 

.658 

.628 

.599 

.569 

.539 

.510 

.481 

.453 
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Si 

15 

16 

17 

18 

19 

MB 

1.000 

gH 

1.000 

6.00 

.999 

1.000 

6.20 

.999 

1.000 

6.40 

.999 

1.000 

6.60 

.999 

.999 

1.  000 

6.80 

.998 

.999 

1.000 

7.00 

.998 

.999 

1.000 

7.20 

.997 

.999 

1.00C 

7.40 

.996 

.998 

.  999 

1.000 

7.60 

.995 

.998 

.999 

1.000 

7.  80 

.993 

.997 

.999 

1.000 

8.00 

.992 

.996 

.998 

.999 

1.000 

8 

9 

10 

11 

El 

13 

14 

1.000 

1.000 

1.000 

1.000 

.999 

1.000 

.999 

1.000 

.998 

.999 

.996 

.999 

.994 

.998 

1.000 

.992 

.997 

.999 

1.000 

.988 

.996 

.999 

1.000 

.984 

.994 

.998 

.999 

1.000 

.979 

.992 

.997 

.999 

1.000 

.972 

.989 

.996 

.999 

1.000 

.964 

.985 

.994 

.998 

.999 

1.000 

.955 

.980 

.992 

.997 

.999 

1.000 

.944 

.975 

.990 

.996 

.999 

1.000 

.932 

.968 

.986 

.995 

.998 

,999 

1.000 

.918 

.960 

.982 

.993 

.997 

.999 

1.000 

.903 

.951 

.977 

.990 

.996 

.999 

1.000 

.886 

.941 

.972 

.988 

.995 

.998 

.999 

.867 

.929 

.965 

.984 

.993 

.997 

.999 

.847 

.916 

.957 

.980 

.991 

.  996 

.999 

.826 

.902 

.949 

.975 

.989 

.995 

.998 

.803 

.886 

.939 

.969 

.986 

.994 

.997 

.780 

.  869 

.927 

.963 

.982 

.992 

.997 

.755 

.850 

.915 

.955 

.973 

.  990 

,996 

.729 

.830 

.901 

.947 

.973 

.987 

.994 

.703 

.  810 

.887 

.937 

.967 

.984 

.993 

.676 

.788 

.  871 

.926 

.961 

.980 

.991 

.648 

.765 

.854 

.915 

.954 

.976 

.989 

.620 

.741 

.  S35 

.902 

.945 

.971 

.986 

.593 

.717 

.816 

.888 

.936 

.966 

.983 

1 " '' 


TABLE  3.  CUMULATIVE  POISS 


TABLE  4 


CONFIDENCE  LIMITS  FOR  A  PROPORTION 


The  observed  proportion  in  a  random  sample  is  s/n. 

The  table  gives  lower  and  upper  confidence  limits  as  well  as 
two-sided  confidence  intervals.  The  values  under  the 
100(1  -  a/2 )%  lower  and  upper  confidence  limits  form  the 
100(1  -  a)%  confidence  interval.  As  an  example,  suppose 
that  n  =  15  and  s  =  4.  The  95%  lower  confidence  limit  for 
p,  the  true  proportion,  is  0.097  and  the  95%  upper  confi¬ 
dence  limit  for  p  is  0.511.  Thus  we  are  95%  confident  that 
p  is  greater  than  0.097  and  we  are  95%  confident  that  p  is 
smaller  than  0.511.  However  we  are  only  90%  confident  that 
p  is  between  0.097  and  0.511.  Thus  the  interval  (0.097, 
0.511)  is  a  90%  confidence  interval.  See  Chapter  6  for  a 
detailed  discussion  of  confidence  limits  and  intervals. 
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TABLE  4.  CONFIDENCE  LIMITS  FOR  A  PROPORTION  (CONT.) 


80%  85% 

INTERVAL  INTERVAL 


90%  90% 

Lwr.  Upr. 
limit  Limit 


INTERVAL  INTERVAL  INTERVAL 


97. 5% 
Upr. 
Limit 


99% 

INTERVAL 


0 

.223 

li 

•itl- 

.103 

an 

;!Za 

•247 

.753 

.33? 

.329 

.424 

.897 

.525* 

.954- 

.63a 

.992 

.772 

1 

0 

.308 

.003 

.445* 

.025  + 

.SS6 

.03? 

.452 

.122 

.738 

.137 

.813 

.232 

.878 

.343 

.933 

•m 

.975- 

.997 

.392 

1 

0 

.194 

.005 

.309 

.038 

.408 

.065- 

.4?a 

u4o 

.501 

.202 

.458 

.249 

.731 

.798 

.419 

.840 

.502 

.915* 

.592 

.962 

.691 

.994 

.804 

1 

.461 

.994 

.779 

9 

.  0 

.162 

.008 

,268 

.04  2 

.360 

.c'ja 

,444 

.14." 

,523 

.20-. 

.598 

.264 

.669 

.331 

.736 

.422 

.799 

.fiVT 

.554 

isa 

.912 

.64o 

.958 

.732 

.992 

.638 

1 

0 

.152 

0 

.169 

.007 

■251 

iO06 

.270 

•039 

.C9l 

.337 

.033 

.350 

.417 

.072 

.438 

.492 

.119 

.513 

.563 

.171 

.583 

.243 

.631 

.227 

.650 

.305- 

.695* 

.286 

.714 

.369 

.757 

.350 

OS 

.437 

.815- 

.417 

.508 

.869 

.487 

.881 

.583 

,919 

.562 

.928 

,663 

.961 

.64? 

.967 

:&9 

.993 

1 

as 

.991* 

1 

0 

,280 

.001 

.389 

.011 

.470 

.557 

.627 

.102 

.692 

.146 

.751 

.195 

.249 

.308 

.805  + 

:8s! 

.373 

.936 

.967 

.989 
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TABLE  4.  CONFIDENCE  LIMITS  FOR  A  PROPORTION  (CONT.) 


80%  85%  90%  "  95%  98%  99% 

INTERVAL  INTERVAL  INTERVAL  INTERVAL  INTERVAL  INTERVAL 


90%  90' 

Lwr.  Upr. 
Limit  Limit 


0 

.160 

.006 

.195 

.030 

•>69 

.06.’ 

.336 

.’.01 

.396 

.146 

.455  + 

.165* 

.516 

.\?9 

.56  7 
,660 

..’9 

5'/l 

.360 

.761 

■MJ 

!»sh 

99.5% 

Lwr. 

Limit 


.<10 

.616 

.665’ 

.677 

•  363 

.735- 

.384 

•448 

-790 

5W 

.$16 

.890 

.587 

.933 

.663 

.970 

•7U6 

.fliu 

.995- 

1 

181 

0 

.613 

0 

.664 

•2V) 

.002 

.J19 

.001 

.368 

363 

.017 

.405- 

.010 

.453 

Cuo 

•Oltf 

.481 

.031 

.569 

•su 

.07' 

•S51 

.059 

.597 

577 

a 

.616 

.094 

.660 

&Uf 

.1*3 

■b,n 

-J  35- 

.718 

7C 

75o 

.734 

.787 

.179 

.629 

.771 

.661 

.809 

.858 

•  666 
.340 

.865* 

.906 

903 

.449 

.966 

.40? 

.941 

943 

.519 

.957 

.471 

.969 

.976 

•595* 

.983 

.547 

.990 

997 

.681 

.998 

.636 

.999 

1 

./<•> 

1 

.736 

0 

.698 

0 

.406 

07 

.486 

.561 

To 

:£aa7 

17 

.744 

59 

.795- 

05* 

.841 

56 

.883 

112 

.960 

TABLE  4.  CONFIDENCE  LIMITS  FOR  A  PROPORTION  (CONT.) 


80%  .  85% 

INTERVAL  INTERVAL 


90%  95%  98%  99% 

INTERVAL  INTERVAL  INTERVAL  INTERVAL 


90% 

92. 5% 

92. 5% 

95% 

95% 

97.  5% 

97.~5% 

99% 

99% 

99.  5% 

Upr. 

Lwr. 

Upr. 

Lwr. 

Upr. 

Lwr. 

Upr. 

Lwr. 

Upr. 

Lwr. 

Limit 

Limit 

Limit 

Limit 

Limit 

Limit 

Limit 

Limit 

Limit 

Limit 

0 

.116 

0 

.133 

0 

.161 

0 

.197 

0 

.223 

.006 

.187 

.002 

.207 

.001 

.238 

0 

.277 

0 

.304 

.022 

.250 

.017 

.271 

.012 

.304 

.007 

.346 

.005* 

•  0U7 

.307 

.040 

.329 

.030 

.363 

.022 

«4a4 

.017 

.432 

.078 

.362 

.060 

.384 

.o$U 

.419 

.ola 

.660 

.034 

.408 

,ui 

.415- 

.099 

.437 

•0o2 

.472 

.065* 

.512 

.055  ♦ 

.539 

.146 

.465- 

.132 

.487 

.113 

.522 

.092 

.561 

.080 

.588 

.186 

.514 

.168 

•536 

*U6 

.570 

.122 

„6oa 

.108 

.6  34 

.223 

■561 

.206 

.581 

.181 

.616 

.155- 

.653 

.138 

*677 

.263 

.607 

.245- 

.620 

.218 

.660 

.189 

.695* 

.171 

.718 

.305* 

.652 

.286 

.672 

.257 

.702 

.226 

.736 

.205* 

.758 

.348 

.695- 

.328 

.714 

.298 

.743 

.264 

.774 

.242 

.795 

.737 

.372 

.755* 

.340 

.702 

♦:o5- 

.an 

.282 

.829 

*6^6 

Ml 

•417 

.664 

.794 

.8  32 

.304 

.430 

.619 

.854 

.347 

■392 

as* 

.862 

.892 

.5  JS  * 

.354 

.513 

.868 

.678 

.887 

.439 

.908 

.412 

,920 

.585* 

.839 

.561 

,901 

.528 

.916 

.488 

.935- 

.461 

.945 

.6  38 

.922 

.61.6 

•912 

,581 

.946 

.540 

.959 

.512 

.966 

.693 

.953 

.671 

,960 

,6  37 

.970 

.596 

.978 

.568 

.903 

-7So 

.970 

♦729 

.933 

.696 

.988 

.656 

.993 

.628 

'.995 

.813 

.996 

•793 

.998 

.762 

.999 

.72  3 

1 

.696 

1 

.084 

1 

.867 

1 

.839 

1 

.803 

1 

.777 

1 

•991  16 

.994  17 

•  1  18 

1  19 


.233  0 

•  317  1 

.387  2 


.610  6 

.657  7 

.701  i 

.763  9 

.792  10 

.819  U 

.856  12 

.886  13 

.915*  14 

.962  IS 

.966  16 

.982  17 
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TABLE  4.  CONFIDENCE  LIMITS  FOR  A  PROPORTION  (CONT.) 


80% 

INTERNAL 

85% 

INTERVAL 

90% 

INTERVAL 

95% 

INTERVAL 

98% 

INTERVAL 

99% 

INTERVAL 

90% 

Lwr. 

Limit 

90% 

Upr. 

limit 

92.5% 

Lwr. 

Limit 

32. 5% 

Upr. 

Limit 

95% 

Lwr. 

Limit 

95% 

Upr. 

Limit 

37.  5% 

Lwr. 

Limit 

97. 5% 

Upr. 

Limit 

99% 

Lwr. 

Limit 

99% 

Upr. 

Limit 

99.  5% 

Lwr.  • 

Limit 

99.  5% 

Upr. 

Limit 

s 

a  ■ 

29 

S 

21 

.591 

.832 

.576 

.842 

.557 

.ess- 

.523 

.873 

.493 

.692 

.470 

.904 

21 

22 

23 

.623 

,860 

.613 

.869 

.594 

.881 

•565- 

.603 

.897 

.530 

.914 

.50? 

.924 

22 

*66$  ♦ 

.888 

.651 

.896 

.6  32 

.906 

•920 

.568 

.935- 

,545- 

.944 

21 

24 

.703 

.914 

.690 

.921 

.671 

.930 

.642 

.942 

.608 

,954 

.584 

.961 

24 

25 

.743 

.938 

.730 

.944 

.712 

.951 

.633 

.961 

.650 

.970 

.626 

.976 

25 

26 

.764 

•92? 

.961 

3B- 

.966 

.754 

38 

.726 

.978 

.693 

.985- 

.670 

.988 

26 

3 

.992 

.984 

.798 

.847 

.772 

.992 

.740 

.995- 

.718 

.996 

IS 

.872 

.996 

.861 

.997 

.998 

.822 

.999 

.792 

.353 

1 

.770 

Mi 

1 

.924 

1 

.915- 

1 

.902 

; 

.881 

1 

1 

1 

29 

a  ■ 

30 

0 

1 

0 

.oo4 

.074 
*1  ?!• 

0 

•003 

.083 

►134 

.180 

0 

.00? 

.095  + 
.149 

0 

•  0<r' 

•116 

•172 

0 

0 

.14 2 
.202 

0 

0 

.162 
•  223 

0 

1 

2 

.018 

J06 

.oir  + 

.012 

.195  + 

.o'>9 

•221 

.ooS* 

.004 

.274 

2 

§ 

.037 

.059 

.209 

.249 

.033 

.054 

•222 

•262 

.028 

.047 

.068 

3T. 

.021 

.038 

*265+ 

.307 

.015- 

,028 

.2?9 

.340 

.012 

.021 

.320 

5 

% 

•063 

.287 

.076 

•3P1 

.319 

.056 

.347 

.045- 

.381 

.038 

5 

6 

.109 

.325- 

.101 

.339 

.091 

.357 

.077 

.386 

.063 

.420 

.054 

.44) 

6 

S 

.135-  . 

.361 

.126 

.375  ♦ 
•411 

.115- 

.394 

.099 

.423 

.083 

.457 

.073 

.480 

S 

.162 

.397 

.152 

.160 

38  + 

.123 

.45? 

.lot 

.493 

.091 

.516 

9 

-  .190 

.432 

.179 

.446 

a66 

.147 

.494 

.127 

•527 

•lit 

.550 

9 

10 

.216 

•466 

.207 

.481 

.193 

.499 

.173 

.528 

JL51 

.561 

J.37 

.533 

10 

11 

.248 

.500 

.236 

.514 

.221 

.533 

.199 

.561 

.176 

.594 

.626 

.160 

.616 

IX 

12 

3S 

ss 

.265  + 

.548 

.250 

.566 

.227 

.594 

.201 

.105  + 

.647 

12 

13 

.295- 

.580 

•27? 

:!S 

.255- 

.626 

.228 

.657 

.687 

.211 

.677 

13 

14 

.333 

.599 

.325  + 

& 

t  106 

.283 

.657 

.256 

33. 

.707 

14 

1? 

.370 

.630 

.356 

.339 

,661 

.313 

.687 

.266 

.716 

.735  + 

15 

16 

.Itol 

•662 

.388 

.675- 

.370 

.692- 

:Si 

.406 

•439 

.717 

.313 

.744 

.293 

.763 

16 

17 

.434 

.692 

•  420 

.705  + 

.402 

.721 

.745  + 

.34) 

.772 

.323 

.840 

iS 

19 

.  18 
19 

.467 

.500 

.723 

.752 

:H§4 

:8f 

IS 

.750 

.779 

.807 

.773 

.801 

at 

20 

.534 

.782 

.519 

.793 

.501 

.472 

.827 

.4?9 

.849 

.417 

.86) 

20 

21 

.568 

.810 

.554 

•t?x 

.535- 

.506 

.853 

.473 

.B7) 

.896 

.450 

.886 

21 

22 

.603 

.813 

•589 

.840 

.570 

.541 

.877 

.507 

.404 

,907 

22 

23 

.639 

.865* 

.625- 

.874 

.606 

.885  + 

.577 

.901 

.Ski 

.917 

.520 

.927 

.946 

2? 

2u 

.675* 

.891 

.661 

.39? 

.663 

.909 

.614 

.923 

.580 

.937 

38 

24 

25 

.713 

.917 

.699 

.924 

,681 

.932 

.653 

.944 

.619 

.955  + 

.962 

25 

26 

.751 

.941 

.730 

.946 

.720 

.953 

.693 

.962 

.660 

.972 
.905  + 

.6)7 

.977 

24 

2i 

ft1 

.876 

.963 

.778 

.820 

.066 

.967 

.761 

.972 

.988 

.735- 

.979 

.702 

.660 

.903 

IS 

24 

.982 

.905- 

.805- 

3B 

.992 

.748 

.995- 

.726 

.996 

20 

.996 

.997 

.851 

.998 

.999 

.798 

.858 

1 

33 

1 

29 

30 

.926 

1 

.917 

1 

.90s- 

1 

.884 

1 

1 

1 

30 
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TABLE  5 


CHI  SQUARE  TABLE 


The  body  of  the  table  contains  values  of 

v2 

xA,v* 

The  table  is  indexed  horizontally  by  A  and  vertically  by 
even  values  for  v.  The  parameter  v  is  a  function  of  the 
number  of  failures  (r)  and  the  parameter  A  is  a  function 
of  the  risk  (a) . 

Example.  The  number  of  failures  (r)  is  13  and  Of  is  C.05. 
find 


Xl-a,2r+2  * 

Since  v  =  2r  +  2  and  r  =  13,  V  =  28.  The  parameter  A  is 
1  -  Of  or  0.95.  Using  the  table  directly,  we  obtain  that 

X0 .95 ,28  =  1^'92  ‘ 

For  degrees  of  freedom  greater  than  100  use 

4, A  *  °-5^  +  ■ 

Values  of  are  given  in  Table  2. 
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TABLE  5.  CHI  SQUARE  TABLE 


vs  ^(gifi*#'iftW#Np4)AsrOm>i‘HNC>NNONmf-ONfnfnnHOU>H^O'»,«p0rg-Tifi«MAin^NHq9O(n0* 
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OPJlANomif\NON»ft^'ONU>N^fsJ'r  C^^'T'OcDHrOlACO  O  N  IH  N  9>  H  O  «  H 
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rt  H  p4  H 
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«®OmuMDONlftM>rv^  «  0  H  M  A  R  O  N  IH  N91 


^0‘O-‘HrtO0<0NfnN^>rBO-<00lf'NO^N0m'A'0t-®©N<0lAmH©irH®^(Mft0^^rtNH<f 
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U>0>rvjtf'»^P^O«Hf0OBHrt<0«0rtmiA<DONIf>Sff'f'J*»,^(^HWlfi©ON“'NOH-r'C»On«\fl-ON*r 

M4HHNNNNmfn^m^4'^>firMrHMr><O«'O<C«NNNh>iS«0a>O>l>99O>OOOOHp4^i^HtNN 

H  H  H 

OffllC'flff'lf'Nin^Nf^ONM'OI>H(NjMO®>OWOOHN-Winc>Nlfl®Ofyj^ir>ONNr-NNS,Ciri^fnH^ 

^SO',,'^®'-,'T"Co)0«iArvO^>fNO^^'ClCDOfniANC>(\J4'-OOMninfvO(M^>0®OW^SO,'"<1*''«* 

HH(-IH(VjNNNmromm4'ir4'^sMrHrilAtf><A<fi4^4NNNN«)Q3B«nO>9lO>900000HHHH 

fsia^w'^^ir>N^-J,0'OOw^h-©ioeJ3ifim-JO^fvjCMr>Hh-rgNM^H/iOMNOfn10(?‘^iAr-C>Msr-fi 

<vj(Mr«O^c>H'fNOfnin«)ON<4>OooN</>O0OHm'r<OttOHN4-inNi)^HAj4>iri<0*«O>o^m'rir\<A 

mu>»>4miriaDON^S<>H'r'0«>ONirt^O>Hf«iu>K)ON^^«H^trtN>^m<OfloON<<>o«Hn^h>OH 
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M  M  M  M  **  M 
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^,»s.-<_4,-*tH<\j<\i<\jf\jf\jfr)f<ir*r*>nrn*r.r»r'J-»jriAu'\inir<miro.£).£5,£stfh*h-NNr-oc'©<»®cc' 

H^ooooo>HO>fsr'Oi>i*,Of-^*j-«f^f-^N^o^O''ro>Of»)OMr'mMOo>a'i»it)Bcjoo'CN^j'0 

NCN^,Cf»lM^e<|’0>ONO'OPjp''flmgjN4-NO''<ltrHaN*/,NOMOnHO'0«>'rJOC'0'r(MHCMri^ 

M  *J  t’*  <T  *£)  N  ^Ord*riT\f^®Ot\Jf<ll^t^O  O  M  ^■lAN^HM'f^aiO^ninNCOON.r  ^NOMfnin'CWOM 

mmtf^}»a<o©vOf'  f-  S  S  N  &  e: 

c;  h  n  r«  ^  n  h  ^CMn'TsMi/NO^  »0‘Oeo»-H*ro^ONO^roooC'^'0,o«^'#-oC‘CJw>o,»y><c«'^  ru;  J1  o  >c  ^ 

»-lN>flNCMJ*01^ttnonO  NTO>£)fVJ«ClO»-^h«#‘*Hf^^,rH«C  H  ID  U\  ^  P  N  sT  *-<  O  *£ro— ICCnnCCOtr  >1  C 

*-»  f\j  m  ir.  >CNC  CiNKiift-ouOHr^iCttC'HrOkr^yo  ^  m  ^  o  O  n  m  in  O  c  c  fJ  n  i’,  f-  c  C  n  xr  £r^ 

V*.  CtJ-K'xfOfJOHCOO'lOcO'-OWNNO'iU'Mft^OUJO^hNN^tClf,  fjCC»Oif.  ■/rWfn«i-in'OWCM 
©«*‘fsj*H«M*/’‘CO'<\)U‘.  aj-ton,\Sfj(cnco<i,0'lriHNncfiftfQ^^HN^OS  ,r  h  I-  ^  h  w  in  f  j  g  «c  m  o  t*“. 

rxj  m  x^  IT  lfl«(?O{\)Mir>'0/0Cf^N^tf'S0  Q  M  m  m  N  C  O  (>J  n  h-  O  c!  ^  :,"i  r-  (O  C?  f J  i‘U’1  h  C'  o  *J  f 
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TABLE  6 


EXPONENTIAL  TEST  PLANS  FOR  STANDARD  DISCRIMINATION  RATIOS 


True 

Decision 

Risks 


Discrimination 


Ratio 


Test 

Duration 
Multiplier  (M) 
T  =  MG, 


Accept-Reject 

Failures 

Reject  Accept 
(Equal  (Equal 


or  More)  or  Less) 


IXC 

12.0% 

9.9% 

1.5 

45.0 

37 

36 

xc 

10.9% 

21.4% 

1.5 

29.9 

26 

25 

XIC 

17.8% 

22.1%  . 

1.5 

21.1 

18 

17 

XIIC 

9.6% 

10.6% 

2.0 

18.8 

14 

13 

XIIIC 

9.8% 

20.9% 

2.0 

12.4 

10 

9 

XIVC 

19.9% 

21.0% 

2.0 

7.8 

6 

5 

xvc 

9.4% 

9.9% 

3.0 

9.3 

6 

C 

J 

XVIC 

10.9% 

21.3% 

3.0 

5.4 

4 

3 

XVI1C 

17.5% 

19.7% 

3.0 

4.3 

3 

2 

XIXC 

28.8% 

31.3% 

1.5 

8.0 

7 

6 

xxc  ■ 

28.8% 

28.5% 

2.0 

3.7 

3 

2 

XXIC 

30.7% 

33.3% 

3.0 

1.1 

1 

0 

^ 'A  'c  'iijAtv:  ■  [ 


Mf: 


I 


I: 


I 

£ 


p- 


K 
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TABLE  7 


SUPPLEMENTAL  EXPONENTIAL  TEST  PLANS 


Teat 

Plan 

No. 

Failures 

Test  Duration 
Multiplier  (M) 

Discrimination  Ratio  6-/6. 
For  Producer's  Rise 

Mo's 

Acc, 

Re.i . 

i  *  no  j 

30J 

20% 

m 

10  Percent 

Consumer's  Risk  (0)  Test  Plana 

10-1 

0 

1 

2.30 

6.46 

10.32 

21.85 

10-2 

1 

2 

3.8-9 

3.54 

4.72 

7.32 

10-3 

2 

3 

5.32 

2.78 

3.47 

4.83 

10-4 

3 

4 

6.68 

2.42 

2.91 

3.83 

10-5 

4 

5 

7.99 

2.20 

2.59 

3.29 

10-6 

5 

6 

9.27 

2.05 

2.38 

2.94 

10-7 

6 

7 

10.53 

1.95 

2.22 

2.70 

10-8 

7 

8 

11.77 

1.86 

2.11 

2.53 

10-9 

8 

9 

12.99 

1.80 

2.02 

2.39 

10-10 

9 

10 

14.21 

1.75 

1.95 

2.28 

10-11 

10 

11 

15.41 

1.70 

1.89 

2.19 

10-12 

11 

12 

16.60 

1.66 

1.84 

2.12 

10-13 

12 

13 

17.78 

1.63 

1.79 

2.06 

10-14 

13 

14 

18.96 

1.60 

1.75 

2.00 

10-15 

14 

15 

20.13 

1.58 

1.72 

1.95 

10-16 

15 

16 

21.29 

1.56 

1.69 

1.91 

10-17 

16 

17 

22.45 

1.54 

1.67 

1.87 

10-18 

17 

18 

23.61 

1,52 

1.62 

1.84 

10-19 

18 

19 

24.75 

1.50 

1.62 

1.81 

10-20 

19 

20 

25.90 

1.48 

1.60 

1.78 

20  Percent  Consumer's  Rink  ((J)  Teet  Plans 


20-1 

0 

1 

1.61 

4.51 

7.22 

15.26 

20-2 

1 

2 

2.99 

2.73 

3.63 

5.63 

20-3 

2 

3 

4.28 

2.24 

2.79 

3.b8 

20-4 

3 

4 

5.51 

1.99 

2.40 

3.16 

20-5 

4 

5 

6.72 

1.85 

2.17 

2.76 

20-6 

5 

6 

7.91 

1.75 

2.03 

2.51 

20-7 

6 

7 

9.07 

1.68 

1.92 

2.33 

20-8 

7 

8 

10.23 

1.62 

1.83 

2.20 

20-9 

8 

9 

11.38 

1.57 

1.77 

2.09 

20-10 

9 

10 

12.52 

1.54 

1.72 

2.01 

20- 11 

10 

11 

13.65 

1.51 

1.67 

1.94 

2(  12 

11 

12 

14.78 

1.48 

1,64 

1.89 

20-13 

12 

13 

15.90 

1.46 

1.60 

1.84 

20-14 

13 

14 

17.01 

1.44 

1.58 

1.80 

20- IS 

14 

15 

18.12 

1.42 

1.55 

1.76 

20-16 

15 

16 

19.23 

1.40 

1.53 

1.73 

20-17 

16 

17 

20.34 

1.39 

1.51 

1,70 

20-18 

17 

18 

21.44 

1.38 

1.49 

1.67 

20-19 

18 

19 

22.54 

1.37 

1,48 

1.65 

20-20 

19 

20 

73.63 

1.35 

1.46 

1.63 

30  Percent 

Consumer's  Risk  (p) 

Test  Plans 

30-1 

0 

1 

1.20 

3.37 

5.39 

11.43 

30-2 

1 

2 

2.44 

2.22 

2.96 

4.59 

30-3 

2 

3 

3.62 

1.89 

2.35 

2.28 

30-4 

3 

4 

4.76 

1.72 

2.07 

2.73 

30-5 

4 

5 

5.89 

1.62 

1.91 

2.43 

30-6 

5 

6 

7.00 

1.55 

1.79 

2.22 

30-7 

6 

7 

8.11 

1.50 

1.71 

2.08 

30-8 

7 

8 

9.21 

1.46 

1.65 

1.98 

30-9 

8 

9 

10.30 

1.43 

!  .60 

1.90 

30-10 

9 

10 

11.39 

1.40 

1.56 

1.83 

30-11 

10 

11 

12.47 

1.38 

1.53 

1.78 

30-12 

11 

12 

13.55 

1 ,  J6 

1,50 

1.73 

30- 13 

12 

13 

14.62 

1.34 

1.48 

1.69 

30-14 

13 

14 

15.69 

1.33 

1.45 

1.66 

30-15 

14 

15 

16.76 

1.31 

1,43 

1.63 

30-16 

15 

16 

17.33 

1.30 

1.42 

1.60 

30-17 

16 

17 

18.90 

1.29 

1.40 

1.58 

30-18 

17 

18 

19.96 

1.28 

1.39 

1.56 

30-19 

18 

19 

21.02 

1.27 

1.38 

1.54 

30-20 

19 

20 

22.08 

1,27 

1.36 

1.52 

NOTEi  The  "Acctpteble  Observed  KTBF"  column  found  in  H1L-STD-781C  bo*‘ bean  deleted 
because  a  small  but  critical  roundinj  error  ukea  them  lucoaaiatent  with  the 
accept/reject  criteria  la  the  "Ho,  Failurss"  column. 


fl-6/* 


TABLE  S 


EXPONENTIAL  CONFIDENCE  LIMIT  MULTIPLIERS  FOR  MTBF' 


Table  8  contains  multipliers  used  to  calculate  confidence 
intervals  and  limits  for  MTBF.  The  table  entries  are 
based  on  an  exponential  model  of  system  failures.  Table 
8a  applies  to  time  terminated  tests  and  Table  8^  applies 
to  failure  terminated  tests. 

Example 

Given  the  following  test  information: 

T  =  1000  hours  (time  terminated) 
r  =  4  failures 
A  1000 

6  =  =  250  MTBF 

Find:  90%  upper  and  lower  confidence  limits  and  80%  con¬ 
fidence  interval  for  MTBF. 

Note:  Test  is  time  terminated;  therefore,  use  Table  8a. 

Multiplier  for  90%  upper  limit  =  2.293 
Multiplier  for  90%  lower  limit  =  0.500 

Upper  Limit  =  0.  =  (2.293) (250)  =  573.25  MTBF 
Lower  Liwit  =  01  =  (0-500(250)  =  125.00  MTBF 

Consequently,  an  80%  confidence  interval  is 
0L  <  0  <  6U 


125.0  <  6  <  573.25  MTBF 
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TABLE  8a 


EXPONENTIAL  CONFIDENCE  LIMIT  MULTIPLIERS  FOR  MTBF 
(Time  Terminated  Testing) 


Total  No. 
of  Failures 

40% 

Interval 

60% 

Interval 

80% 

Interval 

70% 

Lower 

Limit 

70% 

Upper 

Limit 

80% 

Lower 

Limit 

.  80% 

Upper 

Limit 

90% 

Lower 

Limit 

90% 

Upper 

Limit 

1 

0.410 

2.804 

0.334 

4.481 

0.257 

9.491 

2 

0.553 

1.823 

0.467 

2.426 

0.376 

3.761 

3 

0.630 

1.568 

0.544 

1.954 

0.449 

2.722 

4 

0.679 

1.447 

0.595 

1.742 

0.500 

2.293 

5 

0.714 

1.376 

0.632 

1.618 

0.539 

2.055 

6 

0.740 

1.328 

0.661 

1.537 

0.570 

1.904 

7 

0.760 

1.294 

0.684 

1.479 

0.595 

1.797 

8 

0.777 

1.267 

0.703 

1.435 

0.616 

1.718 

9 

0.790 

1.247 

0.719 

1.400 

0.634 

1.657 

10 

0.802 

1.230 

0.733 

1.372 

0.649 

1.607 

11 

0.812 

1.215 

0.744 

1.349 

0.663 

1.567 

12 

0.821 

1.203 

0.755 

1.329 

0.675 

1.533 

13 

0.828 

1.193 

0.764 

1.312 

0.686 

1.504 

14 

0.835 

1.184 

0.772 

1.297 

0.696 

1.478 

15 

0.841 

1.176 

0.780 

1.284 

0.705 

1.456 

16 

0.847 

1.169 

0.787 

1.272 

0.713 

1.437 

17 

0.852 

1.163 

0.793 

1.262 

0.720 

1.419 

18 

0.856 

1.157 

0.799 

1.253 

0.727 

1.404 

19 

0.861 

1.152 

0.804 

1.244 

0.734 

1.390 

20 

0.864 

1.147 

0.809 

1.237 

0.740 

1.377 

30 

0.891 

1.115 

. 

0.844 

1.185 

0 . 783 

1.291 
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TABLE  8b 


EXPONENTIAL  CONFIDENCE  LIMIT  MULTIPLIERS  FOR  MTBF 
(Failure  Terminated  Testing) 


Total  No. 
of  Failures 

40% 

Interval 

60% 

Interval 

80% 

Interval 

70% 

Lower 

Limit 

70% 

Upper 

Limit 

80% 

Lower 

Limit 

80% 

Upper 

Limit 

90% 

Lower 

Limit 

90% 

Upper 

Limit 

1 

0.801 

2.804 

0.621 

4.481 

0.434 

9.491 

2 

0.820 

1.823 

0.668 

2.426 

0.515 

3.761 

3 

0.830 

1.568 

0.701 

1.954 

0.564 

2.722 

4 

0.840 

1.447 

0.725 

1.742 

0.599 

2.293 

5 

0.849 

1.376 

0.744 

1.618 

0.626 

2.055 

6 

0.856 

1.328 

0.759 

1.537 

0.647 

1  904 

7 

0.863 

1.294 

0.771 

1.479 

0.665 

1.797 

8 

0.869 

1.267 

0.782 

1.435 

0.680 

1.718 

9 

0.874 

1.247 

0.796 

1.400 

0.693 

1.657 

10 

0.878 

1.230 

0.799 

1.372 

0.704 

1.607 

11 

C.882 

1.215 

0.806 

1.349 

0.714 

1.567 

12 

0.886 

1.203 

0.812 

1.329 

0.723 

1.533 

13 

0.889 

1.193 

0.818 

1.312 

0.731 

1.504 

14 

0.892 

1.184 

0.823 

1.297 

0.738 

1.478 

15 

0.895 

1.176 

0.828 

1.284 

0.745 

1.456 

16 

0.397 

1.169 

0.832 

1.272 

0.751 

1.437 

17 

0.900 

1.163 

0,836 

1.262 

0.575 

1.419 

18 

0.902 

1.157 

0.840 

1.253 

0.763 

1.404 

19 

0.904 

1.152 

0.843 

1.244 

0.767 

1.390 

20 

0.906 

1 . 147 

0.846 

1.237 

0.772 

1.377 

30 

0.920 

1.115 

0.870 

1.185 

0.806 

1.291 
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TABLE  9 


CONFIDENCE  LIMIT  MULTIPLIERS  FOR  MTBF 
TIME  TERMINATED  RELIABILITY  GROWTH  TESTS* 


The  table  gives  lower  and  upper  confidence  limit  multi¬ 
pliers  which  can  also  be  used  as  multipliers  for  two-sided 
confidence  intervals.  The  table  is  indexed  horizontally 
by  level  of  confidence  and  vertically  by  number  of  fail¬ 
ures  observed.  To  obtain  the  desired  confidence  limit,  we 
take  the  product  of  the  appropriate  multipler  and  the 
point  estimate  of  MTBF. 

When  N,  the  total  number  of  failures,  is  large  (>  100),  we 
may  use  a  normal  approximation  to  obtain  a  multiplier. 
Specifically, 

Lower  Limit  Multiplier  for  10Q(l-Of)%  Confidence 
=  (1  +  za/V2N)"2. 

Upper  Limit  Multiplier  for  100(l-a)%  Confidence 
=  (1  -  za/V2N)‘2. 

Lower  and  upper  limit  multipliers  for  a  100(l-a)%  confi¬ 
dence  interval  are,  respectively 

(1  +  sa/2/V 2Nf2 

and 

a  -  za/2/m'2 

Values  of  and  can  be  in  Appendix  B,  Table  2. 


*Multipliers  for  failure  terminated  tests  are  not  included 
in  this  text. 
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TABLE  9 


CONFIDENCE  LIMIT  MULTIPLIERS  FOR  MTBF 
TIME  TERMINATED  RELIABILITY  GROWTH  TEST* 


80% 

Interval 

90% 

Interval 

95% 

Interval 

98% 

Interval 

of  Failures 

90% 

Lower 

Limit 

90% 

Upper 

Limit 

95% 

Lower 

Limit 

95% 

Upper 

Limit 

97.5% 

Lower 

Limit 

97.5% 

Upper 

Limit 

99% 

Lower 

Limit 

99% 

Upper 

Limit 

2 

0.261 

18.660 

0.200 

38.660 

0.159 

78.660 

0.124 

198.700 

3 

0.333 

6.326 

0.263 

9.736 

0.217 

14.550 

0.174 

24.100 

4 

0.385 

4.243 

0.312 

5.947 

0.262 

8.093 

0.215 

11.810 

5 

0.426 

3.386 

0.352 

4.517 

0.300 

5.862 

0.250 

8.043 

6 

0.459 

2.915 

0.385 

3.764 

0.331 

4.738 

0.280 

6.254 

7 

0.487 

2.616 

0.412 

3.298 

0.358 

4.061 

0.305 

5.216 

8 

0.511 

2.407 

0.436 

2.981 

0.382 

3.609 

0.328 

4.539 

9 

0.531 

2.254 

0.457 

2.750 

0.403 

3.285 

0.349 

4.064 

10 

0.549 

2.136 

0.476 

2.575 

0.421 

3.042 

0.367 

3.712 

11 

0.565 

2.041 

0.492 

2.436 

0.438 

2.852 

0.384 

3.441 

12 

0.579 

1.965 

0.507 

2.324 

0.453 

2.699 

0.399 

3.226 

13 

0.592 

1.901 

0.521 

2.232 

0.467 

2.574 

0.413 

3.050 

14 

0.604 

1.846 

0.533 

2.153 

0.480 

2.469 

0.426 

2.904 

15 

0.614 

1.800 

0.545 

2.087 

0.492 

2.379 

0.438 

.  2.781 

16 

0.624 

1.759 

0.556 

2.029 

0.503 

2.302 

0.449 

2.675 

17 

0.633 

1.723 

0.565 

1.978 

0.513 

2.235 

0.460 

2.584 

18 

0.642 

1.692 

0.575 

1.933 

0.523 

2.176 

0.470 

2.503 

19 

0.650 

1.663 

0.583 

1.893 

0.532 

2.123 

0.479 

2.432 

20 

0.657 

1.638 

0.591 

1.858 

0.540 

2.076 

0.488 

2.369 

21 

0.664 

1.615 

0.599 

1.825 

0.548 

2.034 

0.496 

2.313 

22 

0.670 

1.594 

0.606 

1.796 

0.556 

1.996 

0.504 

2.261 

23 

0.676 

1.574 

0.613 

1.769 

0.563 

1.961 

0.511 

2.215 

24 

0.682 

1.557 

0.619 

1.745 

0.570 

1.929 

0.518 

2.173 

25 

0.687 

1.540 

0.625 

1.722 

0.576 

1.900 

0.525 

2.134 

26 

0.692 

1.525 

0.631 

1.701 

0.582 

1.873 

0.531 

2.098 

27 

0.697 

1.511 

0.636 

1.682 

0.588 

1.848 

0.537 

2.068 

28 

0.702 

1.498 

0.641 

1 . 664 

0.534 

1.825 

0.543 

2.035 

29 

0.706 

1.486 

0.646 

1.647 

0.599 

1.803 

0.549 

2.006 

30 

0.711 

1.475 

0.651 

1.631 

0.604 

1.783 

0.554 

1.980 

35 

0.729 

1.427 

0.672 

1.565 

0.627 

1.699 

0.579 

1.870 

40 

0.745 

1.390 

0.690 

1.515 

0.646 

1.635 

0.599 

1.788 

45 

0.758 

1.361 

0.705 

1.476 

0.662 

1.585 

0.617 

1.723 

50 

0.769 

1.337 

0.718 

1.443 

0.676 

1.544 

0.632 

1.671 

60 

0.787 

1.300 

0.739 

1.393 

0.700 

1.481 

0.657 

1.591 

70 

0.801 

1.272 

0.756 

1.356 

0.718 

1.435 

0.678 

1.533 

80 

0.813 

1.251 

0.769 

1.328 

0.734 

1.399 

0.695 

1.488 

100 

0.831 

l _ 

1.219 

0.791 

1.286 

0.758 

2.347 

0.722 

1.423 

Multipliers  for  failure  terminated  tests  are  not  included  in  this  text 


CHART  1.  CUMULATIVE  POISSON  PROBABILITIES 


USE  OF  CHART  1 


General 


The  chart  of  cumulative  Poisson  probabilities,  for  given  values  of  0/T,  gives 
the  probability  of  c  or  fewer  failures.  The  chart  was  constructed  for  con¬ 
venience  of  use  at  the  expense  of  precision.  For  more  precise  evaluations, 
use  the  Cumulative  Poisson  Table  (Appendix  B,  Table  3)  or  equation  8.7 
(Poisson  Distribution  Equation). 

Probability  of  c  or  Fewer  Failures 

To  find  the  probability  of  c  or  fewer  failures,  enter  the  chart  with  G/T  on 
the  horizontal  scale.  Move  vertically  to  the  curve  for  c,  then  move  hori¬ 
zontally  to  read  the  probability  value  on  either  vertical  scale.  This  is  the 
probability  of  c  or  fewer  failures. 

Example :  The  MTBF  of  a  system  is  100  hours.  What  is  the  probability  of  3 

or  fewer  failures  in  200  hours  of  use? 

8  =  100 
T  =  200 

0/T  =  100/200  =  0.5 
c  =  3 


0.5 


The  probability  of  3  or  fewer  failures  is  approximately  0.86. 

Probability  of  Exactly  c  Failures 

To  find  the  probability  of  exactly  c  failures,  find  the  probability  of  c  or 
fewer  and  the  probability  of  c-1  or  fewer.  The  probability  of  exactly  c  is 


i'W-  lifilMh  I'll  In  i  i  \]i  W^: 
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the  difference  between  these  two  probabilities. 

Example:  The  MTBF  of  a  system  is  100  hours.  What  is  the  probability  of 
exactly  3  failures  in  200  hours  of  use? 


6  =  100 
T  =  200 

6/T  =  100/200  =  0.5 
c  =  3 
c-1  =  2 


The  probability  of  exactly  3  failures  is  approximately  0.86  -  0.68  =  0.18. 
Test  Expos. ire  and  Acceptance  Criterion-Continuous  Time  Test 

We  wish  to  find  a  test  exposure,  T,  and  an  acceptable  number  of  failures,  c, 
such  that  the  probability  of  acceptance  is  p  when  0  =  0^  and  1  -  a  when 

0  =  0q.  This  may  be  done  graphically  with  the  use  of  an  overlay. 

On  an  overlay  sheet,  draw  vertical  lines  at  0/T  =  1  and  0/T  =  0^/0^.  Draw 

horizontal  lines  at  probabilities  p  and  1  -  a,  forming  a  rectangle.  Slide  the 
overlay  rectangle  horizontally  until  a  curve  for  a  single  value  of  c  passes 
through  the  lower  left  and  upper  right  corners.  (It  may  not  be  possible  to 
hit  the  corners  exactly.  Conservative  values  of  c  will  havo  curves  that  pass 
through  the  horizontal  lines  of  the  rectangle.)  This  value  of  c  is  the  ac¬ 
ceptable  number  of  failures.  Read  the  value  of  0/T  corresponding  to  the  left 
side  of  the  rectangle.  Divide  6^  by  this  value  to  find  T,  the  required  test 
exposure. 

Example 

We  wish  to  find  the  required  test  exposure,  T,  and  acceptable  number  of  fail¬ 
ures  c;  such  that  when  the  MTBF,  0  =  =  100  hours,  the  probability  of  ac¬ 

ceptance,  P,  will  be  0.20  and  then  6  =  0q  =  300  hours  the  probability  of 
acceptance,  1  -  a,  will  be  0.90. 
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An  overlay  rectangle  is  constructed  as  shown 


Sliding  the  rectangle  to  the  left,  we  find  that  when  c  =  3  the  fit  is  close 
but  slightly  higher  risks  must  be  tolerated.  Going  to  c  =  4,  the  curve  passes 
through  the  horizontal  lines  of  the  rectangle.  At  the  left  of  the  rectangle, 
e/T  =0.14,  so  the  required  test  exposure  is  approximately  100/0.14  =  714 
hours  and  the  acceptance  criterion  is  4  or  fewer  failures. 


PROBABILITY  OF  C  OR  FEWER  FAILURES 


* 
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ACCEPTANCE  CRITERIA  8-4,  8-10 
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AUTOMATIC  TEST  EQUIPMENT  (ATE) (see  Diagnostic  Systems,  Automatic) 
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Definition  of  4-1 
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Inherent  4-3,  4-9,  4-10 
Mathematical  Expressions  of  4-2 
Multi  Mission/Mode  4-7 

Relationship  to  Maintainability/Reliability  3-1 
Simulation  Models  4-8 

BINOMIAL  MODEL  5-4 

Assumptions  Pertaining  to  5-4 
Confidence  Limits,  Calculation  of  7-3,  7-19ff 
Normal  Approximation  7-4,  7-I9ff 
Poisson  Approximation  7-5,  7-17ff 
Hypothesis  Testing  6-4 
Normal  Approximation  5-6,  5-16,  5-18ff 
Point  Estimate  of  Failure  Probability  7-2,  7-19 
Point  Estimate  of  Reliability  7-3,  7-17 
Poisson  Approximation  5-6,  5-16,  5-l8ff 
Probability  Computation  5-13ff,  5-15 
Probability  Plot  5-7 
Test  Design  8-4ff 

Normal  Approximation  8-5,  8-23ff 
Poisson  Approximation  8-6,  8-23ff 
BIT  (BITE) 

Automatic  Fault  Isolation  Capability  (AFIC)  3-6,  3-8 

Characteristics  External  to  3-8 

Development  and  Evaluation  Considerations  3-10 

Percent  Detection  3-6,  3-8 

Percent  False  Alarms  3-6,  3-7,  3-8 

Percent  False  Removals  3-6,  3-7,  3-8 

Percent  Isolation  3-6,  3-8 

Test  and  Evaluation  of  3-12 

COMBINING  DT  AND  0T  DATA  10-6 
COMPUTER,  RELIABILITY  7-28,  7-30ff 

CONFIDENCE  LEVELS  FOR  PREESTABLISHED  RELIABILITY  LIMITS 
Binomial  Model  7-3ff,  7-15ff 

Exponential  Model  7-12ff,  7-30ff,  7-39ff,  7-42ff 
CONFIDENCE  LIMITS  6-2 

Difference  of  Two  Proportions  7-6,  7-23ff 
Failure  Rate,  Calculation  of  7-14 
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CONFIDENCE  LIMITS  (cont'd) 

Lower  6-3 

MTBF,  Calculation  of  7-10ff,  7-30ff,  7-39ff,  7-42ff 
Proportion  of  Failures,  Calculation  of 
Binomial  Model  7-3,  7-19ff 
Normal  Approximation  to  Binomial  7-4,  7-19ff 
Poisson  Approximation  to  Binomial  7-5,  7-17ff 
Ratio  of  Two  proportions  7-7,  7-23ff 

Reliability,  Calculation  of  (Exponential  Model)  7-14,  7-30 
Upper  6-3 

CONFIDENCE  STATEMENTS 
Definition  of  6-1 

Interface  with  Hypothesis  Testing  6-6 
Interpretation  of  6-2ff 

CONSTANT  FAILURE  RATE  ASSUMPTION  7-9,  7-27,  7-28,  8-7 
CONSUMER'S  RISK  6-5,  7-21ff,  8-3ff 
CONTINUOUS  MATHEMATICAL  MODEL  5-3 
Exponential  (see  Exponential) 

Poisson  (see  Poisson) 

Uniform  5-4 

CONTINUOUS  TIME  TEST  7-2,  7-8 

Constant  Failure  Rate  Assumption  7-9,  7-27,  7-28,  8-7 
Design  of  8-7ff 

Failure  Pattern  Identification  7-8,  7-27,  7-28 
CRITERIA,  ACCEPTANCE  8-4,  8-10 

DATA  BASE  COMPOSITION 

Age  Dependent  Analysis,  for  10-3 
Combining  DT  and  0T  Data  10-6 
Early  Deployment  Data  10-6 

Field  Data  Retrieval  System  10-6 
Lead-The-Force  10-7 
Growth  Analysis,  for  10-4 
DIAGNOSTIC  SYSTEM,  AUTOMATIC  (see  BIT) 

Definition  of  3-5 
Need  for  3-6 
DIFFERENCE  OF  PROPORTIONS 

Confidence  Limits,  Calculation  of  7-6,  7-23ff 
Point  Estimate,  Calculation  of  7-6,  7-23ff 
DISCRETE  MATHEMATICAL  MODEL  5-1 
Binomial  (see  Binomial) 

Hypergeometric  5-5 
DISCRETE  TIME  TEST  7-2ff 
Binomial  Model  7-2ff 
Design  of  8-4ff 

DISCRIMINATION  RATIO  8-8,  8-32ff 

EARLY  DEPLOYMENT  DATA  RETRIEVAL  10-6 
ESTIMATES 

-JJaximum  Likelihood  6-1 
Point  6-1 
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EXPONENTIAL  MODEL  5-3,  5-9 

Confidence  Limits  for  Failure  Rate  7-14 

Confidence  Limits  for  MTBF  7-10ff,  7-30ff,  7-39ff,  7-42ff 

Confidence  Limits  for  Reliability  7-14,  7-30ff 

Failure  Rate,  Point  Estimate  7-10,  7-30ff 

MTBF,  Point  Estimate  7-9,  7-30ff,  7-42ff 

Reliability,  Point  Estimate  7-10,  7-30ff 

Test  Design 

Graphical  Representation  of  Test  Planning  Parameters  8-10ff 
MIL-STD-105D  and  MIL-HBK-108  8-8 
MIL-STD-781C  8-8ff 

Poisson  Distribution  Equations  8-13ff 
EXPOSURE,  TEST  5-8,  6-6ff,  6-10,  8-32ff 

FAILURE  PROBABILITY,  CONFIDENCE  LIMITS 
Binomial  Model  7-3 
FAILURE  PROBABILITY,  POINT  ESTIMATE 
Binomial  Model  7-2 
p-Hat  7-2 
FAILURE  RATE 

Confidence  Limits,  Calculation  of  7-14 
Constant,  Assumption  7-9,  7-26,  7-28,  5-8,  8-7 
Definition  of  2-2 
Plot  of  7-8,  7-27,  7-28 

Point  Estimate,  Calculation  of  7-30ff,  5-27,  5-29,  7-10 
FAILURE  TERMINATED  TEST  7-1 Iff,  7-42ff 
FAILURES 

Contractually  Chargeable  2-4 
Mission  2-3 

Pattern  Identification  7-8 
System  2-4 

FAULT  (SYNTHETIC),  INSERTION  3-12 
FIELD  DATA  RETRIEVAL  SYSTEM  10-6 
FIXED  CONFIGURATION  TEST  7-1 

GROWTH  (RELIABILITY)  TEST  7-1,  9-1 
Supporting  Data  Base  10-4,  10-11 
GROWTH  (RELIABILITY)  TEST  CONCEPTS 
Growth  Tracking 

Confidence  Limits  for  MTBF  9-7,  9-l4ff 
Definition  of  9-2 

Point  Estimates  of  MTBF  9-7,  9-l4ff 
Supporting  Data  Base  10-4,  10-6ff 
Idealized  Growth 

Curve  Development  9-2,  9-9ff 
Definition  of  9-1 
Duane  Growth  Model  9-2 
Growth  Rate  9-2ff,  9*9 
Planned  Growth 

Curve  Development  9-4,  9-13 
Definition  of  9-2 
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HYPOTHESIS  TESTING 

Acceptance  Criteria  8-4 
Binomial  Model  6-4,  7-21ff 
Interface  with  Confidence  Statement  6-6 
Procedure  6-4 

INDEPENDENCE  OF  SUBSYSTEMS  2-5 

MAINTAINABILITY 

Assessment  Considerations  3-2 
Definition  of  3-1 
Off-System  Indicies  3-5 
Physical  Design  Factors  Affecting  3-2 
Quantitative  Indices  3-3,  3-14 
MAINTENANCE 

Annual  Support,  Cost  of  3-5 
Corrective  3-2 
Definition  of  3-1 
Preventive  3-2 

MAINTENANCE  RATIO  (MR)  3-4,  3-14 
MAXIMUM  LIKELIHOOD  ESTIMATE  6-1 
MAXIMUM-TIME-TO-REPAIR  (MaxTTR)  3-4 
MEAN  TIME  BETWEEN  FAILURE  (MTBF) 

Confidence  Interval,  Calculation  of  7-10ff,  7-30ff,  7-39ff,  7-42ff 
Definition  of  2-1 
For  Poisson  Model  5-8 

Point  Estimate,  Calculation  of  7-9,  7-30ff,  7-42ff 
MEAN-TIME-BETWEEN-MAINTENANCE-ACTIONS  (MTBMA)  3-4,  3-14 
MEAN-TIME-TO-REPAIR  (MTTR) (McT)  3-4,  3-14 

MINIMUM  ACCEPTABLE  VALUE  (MAV)  7-30,  6-5ff,  6-9,  6-10,  7-39,  7-42,  8-lff 
MISSION  FAILURES  2-3 
MISSION  RELIABILITY  7-28,  7-30 
MODELS,  MATHEMATICAL 
Continuous  5-3 

Exponential  5-3,  5-9 
Poisson  5-8 
Uniform  5-4 
Discrete  5-1 
Binomial  5-4 
Hypergeometric  5-5 
MODELS,  SYSTEM  RELIABILITIES 
Functional  2-8 
Mixed  2-7,  2-15,  l0-8ff 
Redundant  2-5,  2-13 

Series  2-5,  2-11,  2-12,  2-13,  2-19,  10-9 
NON- HOMOGENEOUS  POISSON  7-9 

NORMAL  APPROXIMATION  TO  BINOMIAL  5-6,  5-16,  5-l8ff 
Confidence  Limit  Calculations 

Difference/Ratio  of  Proportions  7-6ff,  7-23ff 
Proportion  of  Failures  7-4,  7-19ff 
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NORMAL  APPROXIMATION  TO  BINOMIAL  (cont'd) 

Test  Design  8-5,  8-23ff 
NULL  HYPOTHESIS  6-4ff 

OPERATING  CHARACTERISTICS  CURVE 
Construction  of 

for  Binomial  Model  8-19ff 
for  Poisson  Model  8-19,  8-37 
Definition  of  8-17 

PLOT,  FAILURE  RATE  7-8,  7-27,  7-28 
POINT  ESTIMATE 

Definition  of  6-1 

Difference  of  Proportions  7-6,  7-23ff 
Failure  Rate  (Exponential)  7-10,  7-30ff 
Maximum  Likelihood  6-1 
MTBF  (Exponential)  7-9,  7-30ff,  7-42ff 
Proportion  of  Failures  (Binomial)  7-2,  7-19 
Ratio  of  Proportions  7-6,  7-23ff 
Reliability  7-3,  7-9,  7-30ff 
Unbiased  6-1 

POISSON  APPROXIMATION  TO  BINOMIAL  5-6,  5-l6,  5-18ff 
Confidence  Limit  Calculation  7-5,  7-17ff 
Test  Design  8-6,  8-23ff 
POISSON  MODEL  5-8 

Assumptions  Pertaining  to  5-8 
Graphical  Solution  Procedure  8-16 
Non-Homogeneous  7-9 
Normal  Approximation  to  5-9 
Probability  Computation  5-27,  5-29 
Probability  Plot  5-10 
PRODUCER'S  RISK  6-5,  7-21ff,  8-3ff 
PROPORTION  OF  FAILURES  (Binomial  Model)  7-2 
Confidence  Limits,  Calculation  of  7-3,  7-19ff 
Point  Estimate  7-2,  7-19 
Ratio/Difference  of  Proportions 

Confidence  Limits,  Calculation  of  7-6,  ?-23ff 
Point  Estimate,  Calculation  of  7-6,  7-23ff 

RATIO  OF  PROPORTIONS 

Confidence  Limits,  Calculation  of  7-7,  7-23ff 
Point  Estimate,  Calculation  of  7-6,  7-23ff 
RECOVERY  TIME  4-6 
REDUNDANCY 
Active  2-6 

Characteristics  of  2-5 

Implications  of  2-7 

Model,  System  (see  Models,  System) 

Passive  2-6 
RELIABILITY 

Allocation  of  2-9,  2-17 
Computer  7-30ff 


5 


RELIABILITY  (cont’d) 

Confidence  Limits,  Calculations  of  (Exponential  Model)  7-14,  7"30ff 

Definition  of  2-1 

Function  5-11,  5-25,  5-28 

Growth  (see  Growth)  9-1 

Incident  Classification  2-3 

Logistics  Related  2-3 

Maintenance/Supply  Related  2-3,  2-13 

Mathematical  Models  of  5-lff 

Mission  Related  2-2,  7-27 

Multi  Mission/Mode  2-20 

Point  Estimate,  Calculation  of  (Binomial)  7-3,  7-17 
Point  Estimate,  Calculation  of  (Exponential)  7-9,  7-30ff 
Preestablished  Limits,  Confidence  Level  for 
Binomial  Model  7-3 
Exponential  Model  7-12ff,  7-30ff 
Repairable  Systems  of  2-1 
System  Models 

Functional  2-8 

Mixed  2-7,  2-15,  10-8ff 

Redundant  2-5,  2-13 

Series  2-5,  2-11,  2-12,  2-13,  2-19,  10-8 
Tests,  Types  of  7-1 
RISK 

Calculation  of,  Binomial  Model  7-21,  8-23ff,  8-28ff 
Calculation  of,  Exponential  Model  8-32ff 
Consumer's  6-5,  6-9,  6-10,  7-21ff,  8~3ff,  8-21ff 
Producer’s  6-5,  6-9,  6-10,  7-21ff,  8-3ff,  8-21ff 

SAMPLE  SIZE  8-4,  8-23ff 

SAMPLING,  LOT  ACCEPTANCE  5-5,  5-15ff 

SPARES  DEMAND,  UNSCHEDULED  2-4 

SPECIFIED  VALUE  6-5ff,  6-9,  6-10,  7-30,  7-39,  7-42,  8-lff 
STATISTICAL  INDEPENDENCE  2-5 
SYSTEM  FAILURES  2-4 
SYSTEM  MODEL 

Series  (see  Models;  Reliability) 

TEST-ANALYZE-FIX-TEST  (TAFT)  9-1,  9-9 
TEST  DESIGN 

Consumer's  and  Producer's  Risk  (see  Risk) 

For  Binomial  Model  8-4f£,  8-21ff,  8-23ff,  8-28ff 
For  Exponential  Model  8-7ff,  8-32,  8-33ff,  8-37ff 
MIL-STD-105D  and  MIL-HBK-108  8-8 
MIL-STD-781C  8-8ff,  8-32 
Poisson  Distribution  Equations  8-13ff 

Graphical  Poisson  Solution  Procedure  8-16 
Upper  and  Lower  Test  Values  (see  Test  Values) 
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TEST  EXPOSURE 
Adequate  6-6 
Definition  of  6-7 
Excessive  6-7 
Inadequate  6-7 

Relationship  to  Confidence  Intervals  6-6,  6-7 
Total  5-8,  6-6ff ,  6-10,  8-32ff 
TESTS 

Continuous  Time  7-2,  7-8,  8-7 
Discrete  Time  7-2,  8-4, 

Failure  Terminated  7-llff.  7-42£f 
Fixed  Configuration  7-1 
Growth  7-1 

Time  Terminated  7-llff,  7-30ff,  7-39ff 
TEST  VALUES 

Lower  (MAV)  6-5ff,  6-9,  6-10,  7-30,  7-39,  7-42,  8-lff,  8-21ff 
Upper  (SV)  6-5ff ,  6-9,  6-10,  7-30,  7-39,  7-42,  8-lff,  8-21ff 

TIME 

Administrative  and  Logistics  Down  4-1 
Operating  4-1,  4-2 
Recovery  4-6 

Standby  (Warm/Cold)  4-1,  4-2,  4-5 
System  Down  4-1 
System  Off  4-1 
System  Up  4-1 

Terminated  Test  7-llff,  7-30ff,  7-39ff 
Total  (foe  Availability)  4-1,  4-2 
Total  Maintenance  4-1,  4-2 

UNBIASED  ESTIMATE:  6-1 
UNIFORM  MATHEMATICAL  MODEL  5-4 
UNSCHEDULED  MAINTENANCE  ACTIONS  2-3,  2-4 
UNSCHEDULED  SPARES  DEMANDS  2-4 
UPPER  TEST  VALUE  (see  Test  Values) 

Discrimination  Ration  8-8 

ZERO  FAILURE  EVALUATION  7-39ff,  7 -15 if 
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